Skill Development and Knowledge Acquisition Cultivated by Maker Education: Evidence from Arduino-based Educational Robotics

This study investigated elementary school students’ learning performances and behaviors in a maker education program. An informal after-school learning environment entitled Robot MakerSpace was created at a public elementary school in Taiwan and 30 grade 5 students voluntarily participated in a 16-week educational experiment. The student participants were randomly divided into two experimental groups. Students in the maker group received weekly educational robotics lessons, whereas those in the nonmaker group only engaged in other after-school learning activities such as homework practice in traditional classrooms. Mixed methods research was used for data collection. An experiment with a pretest–posttest and control group design was employed to measure the students’ electrical engineering and computer programming content knowledge and problem-solving skills. In addition, a qualitative approach with an emphasis on filed observation was adopted to evaluate the instructional implementation of the maker education program. The quantitative findings revealed that maker education training significantly improved the electrical engineering and computer programming content knowledge of the students and improved their problem-solving skills. The qualitative findings showed the students required considerable learning support from the instructor such as strategies for software and hardware debugging.


Arduino-based Educational Robotics Approach
In recent years, educational robotics such as Lego robots have been integrated into curriculum instruction in after-school programs and clubs (Barak & Zadok, 2009;Sullivan, 2008).In addition, because of the rapid development of robot technologies, particularly programming tools and low-cost electronic kits, educational robotics products constitute more accessible tools for school students (Horizon Report, 2016) and are more compatible with the concept of maker education (Alimisis, 2013).According to the analytical results of a systematic literature review of educational robotics conducted by Benitti (2012), previous studies have tended to employ Lego robots as primary research tools to facilitate student learning.Furthermore, Benitti suggested that future research should adopt various educational robotics products such as low-cost robots for teaching and learning, thereby rationalizing the use of Arduino-based educational robotics products in maker education programs in the current study.
Although educational robotics provides an innovative pedagogy in informal learning environments (Lieto et al., 2017;Padir & Chernova, 2013), previous studies using Lego robotics products have presented divergent findings regarding content knowledge gains and the development of problem-solving skills.Barker and Ansorge (2006) indicated that students who used only Lego robotics products increased their science and technology content knowledge.Hussain, Lindh, and Shukur (2006) investigated the effects of Lego robotics training on students' learning behaviors and reported no significant improvements in mathematics content knowledge and problemsolving skills among the study subjects.Williams, Ma, Prejean, and Ford (2007) analyzed how using Lego robotics products influenced knowledge acquisition among students.The results indicated that enhanced physical content knowledge but no improvement in scientific inquiry skills.Lindh and Holgersson (2009) observed that 1 year's Lego robotics training yielded noticeable improvements in logical problem-solving in some students.
A little difference existed on the function comparison between Arduino-based and Lego robotics.The technological features of Arduino-based robotics were extremely similar to those of Lego robotics.However, students using Lego robotics required extra Lego programming training while students manipulating Arduinobased only needed Scratch programming, which was a commonly used programming tool in the elementary schools.In addition, compared to previous Lego robotics studies (e.g., one small group for one robot), the low-cost Arduino-based robotics enabled the instructors to adopt one-to-one educational computing initiatives (Bebell & O'Dwyer, 2010) in which students had a learning opportunity to operate their own robots.Because several learning benefits identified in previous Lego robotics research, the Arduino-based robotics may have a potential to support student learning in different cognitive domains.

Contribution of this paper to the literature
• Providing evidence showing that maker education training significantly improved students' content knowledge and might cultivate students' problem-solving skill development.
• Providing a new instructional strategy for implementing a maker education program.
• Identifying that students in the maker group required considerable learning support and continual encouragement from the instructor.

Research Purpose and Questions
The research team of the present study collaborated with the administrative board of one public elementary school in Taiwan to design and develop a maker education program.An informal after-school learning environment entitled Robot MakerSpace was implemented from September 2016 to January 2017.For one semester, grade 5 students voluntarily participated in regular educational robotics training sessions (3 h per week), where they assembled Arduino-based robots from scratch.Through learning by making, the leaners installed various electronic gadgets and sensors to the robot platform and used Scratch programming tool to control each the robot's actions.An experiment with a pretest-posttest and control group design was employed to measure the students' content knowledge of electrical engineering and computer programming, as well as their problem-solving skills.In addition, a qualitative approach was adopted to evaluate the instructional implementation of the program.The primary goal of this study was to investigate elementary school students' learning performances and behaviors while engaged in the maker education program.The following two research questions were devised:  RQ1: Did participation in the maker education program enable the students to acquire more electrical engineering and computer programming content knowledge and developed further problem-solving skills?
 RQ2: What were the students' learning responses to the maker education program?

Theoretical Foundations of Maker Education
Although maker education is a new phenomenon, its theoretical foundations are rooted in the following three older theories: Papert's constructionism theory, Dewey's experiential learning approach, and Montessori's educational methods (Dougherty, 2012;Kurti et al., 2014;Lee, 2015).First, Papert's constructionism theory emphasizes on learning through making.Papert (1993) proposed that learners could construct personal knowledge bases through building meaningful artifacts to represent their unique thinking solutions.In other words, idea generation and knowledge acquisition may occur when learners are actively engaged in producing physical objects.Second, Dewey's experiential learning approach emphasizes learning by doing.Dewey (1997) asserted that students gain valuable learning experience through personal actions.In other words, problem discovery and problem-solving may occur when learning activities are performed, thereby yielding meaningful learning experiences for students.Finally, Montessori's educational methods are based on learning through play.In Montessori's educational environments, young children are allowed to play with several designed materials to understand abstract concepts, thereby implying that direct object manipulation enables the discovery of new knowledge (Lillard, 2003).
The current study adopted learning through making, learning by doing, and learning through play as the theoretical frameworks for educational robotics: 1. Learning through making: Students used one programming tool to control robot actions.During the programming process, students construct their programming knowledge by completing feasible programming works.The robot actions represented their thinking solutions.Overall, learning through making may benefit students' content knowledge acquisition of programming.
2. Learning by doing: When students designed programming patterns, they might build their coding experiences through a process of trial and error.Ultimately, students might gradually obtain debugging principles, resulting in developing problem solving skills.Thus, learning by doing may support students' skill development of problem solving in the study.
3. Learning through play: Under a playful learning atmosphere, students might manipulate various electronic sensors and gadgets in the robot platform.Object manipulation enabled students to explore abstract concepts of electrical engineering.Overall, learning through play may strengthen students' basic understanding of electrical engineering.

Instructional Frameworks of Maker Education
Previous studies have frequently proposed principles for instructional maker education environments rather than specific instructional frameworks.Kurti et al. (2014) asserted that maker education should emphasize the following six principles: curiosity invitation, wonder inspiration, playfulness involvement, work celebration, failure encouragement, and peer collaboration.Similarity, Lee (2015) proposed that maker education should contain the following four critical elements to cultivate an effective learning atmosphere: playfulness, asset and growth orientation, positive failure encouragement, and collaboration.In addition, because maker education is a To identify learners' engineering perspectives in the context of early childhood education, Bagigati and Evangelou (2016) proposed the following five-step fundamental design process: (a) think, (b) research, (c) create, (d) test, and (e) consult.The consultation stage is similar to the final states of the processes described by Stone-MacDonald et al. (2015) and Bers et al. (2014). Fernandez-Samsca, Barrera, Mesa, andPerez-Holguin (2017) proposed the following four-step engineering design process to emphasize robotics education for preschool students: (a) sensitization, (b) design, (c) construction, and (d) evaluation.In the sensitization stage, students are trained to conceptualize learning task goals.
For the current study, by incorporating various elements into an engineering design process, a three-stage instructional framework (Figure 1) was designed for the maker education program, which was fully compatible with the characteristics of Arduino-based educational robotics.The stages were the pre-design, in-design, and postdesign learning stages.In the first stage, after an instructor has imparted relevant knowledge, the students experiment by installing specific sensors on their robots based on the instructors' programming content.In the second stage, a new learning challenge is assigned to students, where they are responsible for using their imagination to re-design programming patterns for testing on the robot platform.In the final stage, the students serve as peer reviewers to observe the creative ideas of others and subsequently self-evaluate their own works by completing self-reflective journals.

Research Design
This study applied mixed methods research to answer the research questions.Creswell and Clark (2007) indicated that the implementation approach of using quantitative and qualitative models should be properly defined in a mixed methods design.The researcher divided data collection into two stages with an emphasis on the quantitative model.In the first stage, an experimental pre-and posttest control group design (Table 1) was adopted to examine the effect of maker education on elementary school students' engineering and programming knowledge and problem-solving skills.In the second stage, a qualitative field observation method was employed to enable the researcher to gain a deep understanding of maker education.The two stages were conducted concurrently.
In the quantitative-based stage, the independent variable was the type of after-school instructional system and the dependence variables were the students' learning achievements in electrical engineering and computer programming and problem-solving skill development statuses.Before the experiment, students in both groups (Treatment 1 and Treatment 2) underwent three pretests to control the effects of prior knowledge on the experimental results.During the educational experiment, only students in Treatment 1 participated in several learning activities designed for the maker education program.Upon completion of the experiment, both groups underwent several posttests to measure their dependent variables.Regardless of the test purpose type, all assessment tools listed in the pre-and posttest categories were administered within a 40-min period.In the qualitative-based stage, the research team used a video camera to record the students' classroom behaviors.In addition, the researcher recorded field notes to observe student participation in the learning activities and facilitate subsequent informal conversations conducted with the students (without an interview guide) after the completion of each learning session.

Research Participants
Before the current study was conducted, a recruitment sheet was sent to all the grade 5 students at one public elementary school in Taiwan.After a 3-week recruitment process, 30 students were selected for voluntary participation.Subsequently, the participants were randomly divided into two experimental groups.Table 2 presents a profile of the participants.Students in the maker group participated in a 16-week educational robotics training course in maker education program (Robot MakerSpace).By contrast, students in the nonmaker group participated in other after-school learning activities such as homework practice in traditional classrooms.As compensation for their contribution, the nonmaker students were scheduled to participate in a second maker education program in the following semester.

Computer programming test
A programming achievement test was designed to measure the students' understanding of Scratch programming.The test question items were obtained from the test bank of a national Scratch programming competition.The test contained 15 multiple-choice questions and 5 short-answer questions with a score range of 0-100.To ensure quality, a computer science professor, instructional technology professor, and three elementary school teachers with experience of teaching Scratch programming collaborated to review the test content.In addition, a modified version of the test was administered to 25 grade 5 students with experience of learning Scratch programming to verify the appropriateness of each for potential participants.Item analysis verified that the discrimination (> 0.3) and difficulty (0.2-0.8) indices met the acceptable standards (Aiken & Groth-Marnat, 2006).In addition, the KR-21 reliability test revealed a Cronbach's alpha value of 0.85. Figure 2 presents a short-answer question.Problem-solving skill test Chan and Wu (2007) modified the problem-solving test designed by Zachman, Jorgensen, Huisingn and Barrett (1984) to measure problem-solving skills of Chinese elementary school students in grades 4-6.The test comprised the following three psychological constructs: finding causes, finding solutions, and avoiding problems and outlined six scenario-based cases with 15 test items for problem-solving.The score range of was 0-135.During the test, the students had to carefully read scenario descriptions and provide short-answer solutions to the test items.Chan and Wu (2007) reported strong validity and reliability through a nationwide sampling survey.Overall, the reliability coefficient of the test was 0.91.Because of the potential for various answers from different students, Chan and Wu (2007) provided a possible answer sheet to objectively score the test results.

Electrical engineering test
The current study developed a learning achievement test to assess students' understanding of electrical engineering concepts.This summative test comprised 15 multi-choice questions and 1 short-answer question with 5 subtest items.The score range was 0-100 with higher scores representing higher learning achievements in electrical engineering.To verify the accuracy and validity of the questions, an electrical engineering professor and two elementary school teachers of science and technology collaborated to revise the test content.The final version was administered to 30 elementary school students who participated in the maker education pilot study.Item analysis verified that the discrimination (> 0.3) and difficulty (0.2-0.8) indices were within acceptable ranges (Aiken & Groth-Marnat, 2006).In addition, the KR-21 reliability test revealed a Cronbach's alpha value of 0.87. Figure 3 presents a short-answer question.

Design principle of achievement tests
In the study, the computer programming test and the electrical engineering test were used to measure the level of students' knowledge acquisition.During the maker education program, students were only given an opportunity to investigate various electronic sensors, resulting in obtaining a basic understanding of electrical engineering.According to Bloom's revised taxonomy (Anderson et al, 2001), the design principle of the electrical engineering test centered on two cognitive domains: remembering and understanding.A leaning difficulty might appear if the level of electrical engineering moved toward a higher order thinking.However, regarding the computer programming test, because students constantly used Scratch programming to design action patterns of educational robotics, the feasible design principle might emphasize four cognitive domains: remembering, understanding, applying, and analyzing.

Qualitative data
Video clips of learning scenarios, field notes from student observation, detailed summaries of informal conversations with students were the three main sources of qualitative data, thereby fulfilling the data triangulation requirement proposed by Patton (2002) and ensuring qualitative data reliability.

Educational robotics
The educational robotics product used in this study was an Arduino-based robot (mBot v1.1) developed by MakeBlock Co, Ltd.Following one-to-one educational computing initiatives (Bebell & O' Dwyer, 2010), the student participants received personal mBot to experience their learning benefits.A robot platform consists of one Arduino motherboard and various steel frame parts.Tool kits were provided to enable the students to install various electronic sensors and gadgets onto the platform.Once the hardware had been successfully installed, the students could use the Scratch programming language (mBlock: Scratch for mBot) to design anticipated robot actions.Figure 4 shows one student installing electronic sensors and gadgets onto the robot platform.

Robot MakerSpace
Before the study was conducted, the research team and school administration collaborated to transform a traditional classroom into an area suitable for Robot MakerSpace. Figure 5 shows a site plan diagram of the area.The learning environment contained the following four areas: the desk computer, robot testing, electronic supply, and screen display areas.The students conducted programming tasks in the desk computer area and manipulated electronic sensors and tested robots in the robot testing area.An 80-inch large touch TV screen and two projectors displayed instructional content.To cultivate an atmosphere of positive failure encouragement (Lee, 2015), the electronic supply area provided numerous unused electronic components for students who accidentally damaged their electronic sensors and gadgets.In addition to hardware preparation, a training workshop for teaching the instructional principles and frameworks of maker education was held for the two schoolteachers responsible for providing instruction during the educational experiment.

Curriculum Design
In this study, Robot MakerSpace was an after-school maker education program offering a 16-week training curriculum for student participants.Various instructional themes were presented in weekly 3h learning sessions.Table 3 demonstrates the Robot MakerSpace curriculum design.To enable students to become proficient in the programming language, extensive hands-on practice of using Scratch programming was included in the first 2 weeks.Toward the end of the curriculum, an informal robot competition was scheduled to enhance the students' learning experiences and signify completion of the course.Figure 6 shows a robot traveling through a maze pass.
Two instructors (they were also responsible for providing learning support for students in the control group who faced difficulties during homework practice in other after-school learning activity) were employed to teach the Robot MakerSpace curriculum; the principal instructor taught weekly classes and facilitated student learning, whereas the assistant instructor was in charge of distributing electronic components and maintaining classroom discipline.In the initial 3 weeks of the curriculum, the principal instructor expected that the students might be able to direct their own learning with limited instructional guidance.However, several students faced obstacles under this learning model which influenced the overall teaching schedule.Therefore, in the fourth week, to better facilitate robot design and development, key skills regarding computational thinking based on flowchart design and systematic debugging through standard operating procedures were taught to the students.EURASIA J Math Sci and Tech Ed

/ 15
All learning activities in the maker education program were structured by the proposed three-stage instructional framework (pre-design, in-design, and post-design); for example, in the light-emitting diode (LED) unit, students copied programming codes from the instructor and tested the change process of LED lights in the pre-design stage (approximately 40 min).Upon completion of the initial learning practice, a complex design task of improving the teacher's work was assigned.During the in-design stage (approximately 100 min), some students creatively incorporated alternative electronic gadgets such as motors with LED lights to redesign their programming patterns.After developing new solutions for robot actions, in the post-design stage (approximately 40 min), the students were required to review their peers' works through social collaboration and complete selfreflective learning journals.The basic requirement of the self-reflection assignment was to convey what they learned or experienced during the learning sessions.Figure 7 presents a portion of one student's self-reflective journal.

Data Analysis
In the quantitative stage, a t test and multivariate analysis of covariance (MANCOVA) were conducted to evaluate the effects of the independent variable on the students' learning performances.The t test measured changes in the learning process that had occurred throughout the educational experiment.The MANCOVA was Chou / Educational Robotics in Maker Education 10 / 15 conducted to eliminate the influence of students' prior knowledge as a covariant variable in the experimental results (Tabachnick & Fidell, 2007).In the qualitative stage, the qualitative analysis method proposed by Moustakas (1994) was used to interpret qualitative information.Moustakas proposed an analysis model with the following four stages: phrase identification, meaning formulation, theme creation, and text description.During the data mining process, this model enabled the principal researcher to formulate several meanings by identifying key phrases from qualitative data and subsequently create representative themes with detailed text descriptions from the formulated meanings.

Quantitative-based Results
The t-test, and MANCOVA results are summarized in Tables 4 and 5, respectively.The statistical findings indicate that significant learning gains from the three tests were identified only in the experimental group (problem solving test: t = 8.88, p < 0.01; electrical engineering test: t = 11.22,p < 0.01; programming test: t = 14.51, p < 0.01).In other words, the students enrolled in the maker education program generally significantly improved their knowledge of electrical engineering and computer programming and enhanced their problem-solving skills.By contrast, those who participated in traditional after-school clubs exhibited unchanged performances on the three tests after completion of the program.In addition, after excluding the influence of prior knowledge, the students in the maker group significantly outperformed their nonmaker counterparts on all three tests (problem solving test: F = 7.85, p < 0.05; electrical engineering test: F = 147.7,p < 0.01; programming test: F = 79.24,p < 0.01).Therefore, the maker education program seems beneficial for improving students' knowledge of electrical engineering and programming and developing problem-solving skills.

Qualitative-based Results
The following six representative themes related to the students' learning behaviors were identified through qualitative data mining:

Theme 1: Overcoming the fear of failure
Although the instructor had initially informed the students of the purpose of the electronic supply area, the video clips revealed that most students carefully manipulated electronic gadgets and installed electronic sensors onto their robot platforms early in the curriculum.At the research site, the principal researcher perceived that the students feared damaging the electronic components.They constantly asked questions such as "what if my gadgets break" and "will I be punished."In the informal conversations, the students revealed that the teachers would not prepare extra tools or gadgets for them during the learning tasks at school.Consequently, the students were less likely to take risks to make new discoveries.However, at times when the instructor continually stressed the educational principle of the maker education program (i.e., positive failure encouragement), the students were more likely to adventurously investigate the electronic components.

Theme 2: High motivation throughout the class
All the students demonstrated high motivation to engage in the learning process.They paid attention to the instructor and actively followed the class procedures.Their active learning behaviors were frequently documented in the researcher's field notes.However, during the first 3 weeks, a class discipline problem occurred as a result of student excitement.Sounds observed in the video clips such as loud talking frequently disrupted the instructor's teaching pace.Therefore, the instructor had to reiterate a learning rule to regulate the students' behaviors.When asked about the cause of their high motivation, most students praised the engaging curriculum design in comparison to those of traditional after-school activities.One girl stated the following: "If I was not here, I would have been assigned to after-school cram schools by my parents where I always practice boring test papers."

Theme 3: Facing obstacles during the thinking process
Because the students imitated the instructor's actions in the pre-design stage, the classroom learning loads were consistent across all students.However, the in-design stage brought the possibility of several learning obstacles.Video clips recorded in the first 3 weeks frequently showed students staring into space or doodling on paper during the thinking process.Few students were able to rapidly complete the engineering design tasks.Some who were unable to determine solutions sought help from the instructor.In the fourth week, to avoid additional learning problems, the instructors began to offer design advice through flowcharts which facilitated the students' development of computational thinking skills.The researcher observed that after grasping specific design skills, the students required less time to propose thinking solutions.

Theme 4: Debugging learning challenge
In addition to the problem of redesigning the instructors' works, debugging constituted a major learning challenge for the students.When the students experimented with new programming patterns for the robots, spontaneous technical problems often disrupted their learning progress, forcing them to consider "what to do next" or "how to fix it."The video clips show some students spending long periods considering possible solutions.To reduce the cognitive load, the instructor demonstrated standard operating procedures for accurately debugging programming and electronic device problems.Understanding specific problem-solving methods enabled the students to spend less time seeking solutions.One boy stated the following: "I knew the instructor wanted us to try something by ourselves but most of the time, I did not know why the robot was not functioning well…the instructor's tips really helped."

Theme 5: Willing to review but unwilling to self-reflect
Upon entering the post-design stage, all students hesitantly halted their activities despite their facial expressions demonstrating the desire to continue.During the post-design stage, the students were permitted to freely walk around the classroom to review their peers' works within the allotted period.However, several students' enthusiasm for sharing their works led to occasional noisy conversation and meaningless discussion if the instructor did not remind them to stay on task.Although the instructor had previously provided specific reflection guidelines, after engaging in social negotiation, may students were not willing to document their learning processes in detail.Such students had to be encouraged to elaborate on the written descriptions in their self-reflective journals by being shown the learning benefits of reflection; for example, one video clip showed the instructor stating the following: "It (self-reflection) is similar to a test.It allows you to reflect on what you did… You might learn better.You should write more."

Theme 6: Competition as an alternative evaluation activity
In the final 2 weeks of the class, all students focused on the robot competition, which served as an alternative evaluation activity enabling the students to practice the knowledge acquired over the preceding 14 weeks.In the week before the competition, the instructor explained all the competition rules and requirements in detail and the students practiced the programming patterns they had learned and designed solutions for programming their robots to pass through the maze.If the students faced learning problems, the instructor guided them toward correct thinking pathways rather than directly highlighting programming bugs.On the day of the competition, the students took turns in testing their robots in the Styrofoam maze.Some students succeeded in completing the maze, whereas others were disappointed with the dysfunctionality of their robots.After the competition, the instructor established a discussion forum to enable students to review their deficiencies.One boy stated the following: "I really felt disappointed with the result.I thought the robot could pass.After it (review process), I knew that I had forgotten to link one programming code to one sensor."

Discussion
After 16 weeks of educational training, the quantitative analytical results indicated that the students in the maker group outperformed their nonmaker counterparts in acquiring electrical engineering and computer programming knowledge.In other words, students in the maker education program enhanced their conceptual understanding of the content knowledge of two fields through participation in an organized curriculum.Although this study adopted the low-cost educational robotics, the findings echoed the results of previous studies which have reported that using Lego robotics products could improve science and technology content knowledge (Barker & Ansorge, 2006;Williams et al., 2007).In addition, the significant learning gains in the maker groups in the current study may be attributable to the learning atmosphere of the maker education program, which encouraged the students to actively construct personal knowledge domains by building meaningful artifacts (Dewey, 1997;Dougherty, 2012;Papert, 1993).However, because the maker education program did not purposely allow students to prepare test-related information, few students achieved the level of master learning (scoring more than 80 points).Nevertheless, all the students passed the 60-point mark on the electrical engineering and computer programming tests.
In contrast to the two achievement tests (electrical engineering and computer programming), which were directly related to curriculum content, the problem-solving skill test was a learning application designed to evaluate the students' cognitive thinking development.After the educational experiment, the problem-solving skills of the maker group had significantly surpassed those of the nonmaker group.In other words, maker education training seemed to cultivate problem-solving skill development.This finding contradicts that of Hussain et al. (2006), who observed no significant improvements in problem-solving skills among students who participated in Lego roboticsbased learning activities.The educational phenomenon in the current study had two possible causes.First, in the in-design stage, designing flowcharts encouraged the students to use logical reasoning and computational thinking (Bers et al., 2014), such as applying if-then coding statements to facilitate their engineering designs.The second cause was robotics testing technique.Standard operating procedures for debugging possibly enabled the students to gain an in-depth understanding of problem-solving skills.
Taiwanese elementary school students commonly complete school assignments and practice what they have learned by attending after-school cram schools (Liu, 2012).The maker education program offered a more engaging learning alternative to cram school attendance.The innovative curriculum design of the maker education program motivated the students to actively engage in various learning activities such as weekly learning sessions and the robot competition, regardless of difficulty level.However, excitement stimulated by high motivation sometimes created class discipline problems, for which learning rules were required to enable the teacher to gradually regain control in the classroom.In addition, possibly because the students had been enrolled in the traditional educational system for a long time, they might have adopted their existing learning patterns in the new learning environment (Chou & Chen, 2010).Traditional classroom culture does not enable students to manipulate learning devices and use electronic gadgets.Gaining an understanding of the educational principles of maker education could encourage students to investigate the electronic components to make new discoveries.
Although the proposed instructional framework of the maker education program was based on various engineering design processes in STEM education, several learning problems with corresponding solutions appeared in the educational experiment.First, several students were severely delayed in producing engineering design outcomes if the instructor did not provide specific strategies to assist redesign.To tackle this problem, flowchart design could serve as a learning scaffold (Donohue, 2015) to facilitate student learning in the in-design stage.Second, although proposing redesign methods, several students remained engaged in the problem-solving process for a long time.Offering standard operating procedures for debugging could serve as a constructive learning method (Jonassen, 1999) for supporting software and hardware testing.Finally, in the post-design stage, some students completed the basic self-reflection requirement with minimal effort, thereby yielding limited reflection content.Continual encouragement from the instructor could cultivate a growth-oriented learning context (Lee, 2015) where students are more willing to contribute written feedback.

Contribution of the Current Study
This study investigated elementary school students' learning performances and behaviors in a maker education program (Robot MakerSpace).Regarding the first research question, the students in the maker group outperformed EURASIA J Math Sci and Tech Ed 13 / 15 their nonmaker counterparts on two achievement tests and one problem-solving test.In other words, the maker education program seemed to significantly improve the students' electrical engineering and computer programming content knowledge and enhance their problem-solving skills.Regarding the second research question, although the curriculum design of the maker education program motivated the students to actively participate in various learning activities, they required considerable learning support from the instructor such as skills and strategies for producing engineering designs and software and hardware debugging to adapt to the new learning environment.In addition, continual encouragement from the instructor could encourage students to exert more efforts in self-reflection and adventurous investigation.

Research Limitations and Directions for Further Research
The characteristics of the maker education program designed in this study indicate that the research results may be difficult to replicate in the future.This study had several research limitations regarding generalization of the findings.First, the proposed three-stage instructional framework of the maker education program was developed to correspond with educational robotics.Adopting various instructional approaches may yield different learning outcomes.Future studies could modify the framework by excluding the post-design or in-design stage to verify the instructional effectiveness of maker education.Second, although maker education covers several learning domains, because of time constraints, this study only focused on the students' skill development and knowledge acquisition.Future studies could investigate students' creative outcomes such as design works, learning patterns for programming debugging, and self-reflection, all of which provide other learning perspectives from which to examine maker education.Third, the learning scaffolding provided by the instructors in the study benefited students' maker-centered learning.The level of learning support may influence the students' learning processes.Future studies could investigate the learning scaffolding strategies in the maker education program.Finally, the learning context in the study was an after-school maker education program.Whether students would exhibit similar levels of motivation, particularly for problem-solving skills, in school classrooms is yet to be determined.Future studies could investigate the effects of integrating maker education into regular school classes.

Implications for Teaching Practice
Although this study was conducted at an elementary school, the results could be used as a reference for educators in designing maker education programs for other educational levels.First, when the students were first exposed to the maker education program, their excitement sometimes caused class discipline problems.To better facilitate adherence to the learning schedule, appropriate classroom rules should be established to maintain order.Second, insufficient readiness to perform self-direct learning in the maker education environments, particularly self-reflection and adventurous investigation, affected the learning outcomes.Continual encouragement from instructors could guide students to produce positive outcomes.Third, maker education places less emphasis on traditional competence-based tests.If schools wish to apply master learning mechanisms, establishing after-school maker education programs is unnecessary.Finally, developing design skills and strategic debugging procedures are critical aspects of maker education programs.Instructors should emphasize these elements in each learning session to enable students to gradually adjust their learning patterns.
form of STEM education, related engineering design processes may provide references for constructing instructional frameworks in maker education.A curriculum framework for science, technology, and engineering education proposed by Massachusetts Department of Education outlines the following eight-step engineering design process (Massachusetts Department of Education, 2006): (a) identify the need or problem; (b) research the need or problem; (c) develop possible solutions; (d) select optimal possible solution; (e) construct a prototype; (f) test and evaluate the prototype; (g) communicate the solution, and (h) redesign the prototype.To investigate children's computational thinking, Bers, Flannery, Kazakoff, and Sullivan (2014) simplified these eight steps by applying the following simple verbs: ask, imagine, plan, create, test, improve, and share.Stone-MacDonald, Wendell, Douglass, and Love (2015) adopted a similar engineering design process to that of Bers et al. (2014) for incorporation into the following four-step process: (a) think about it, (b) try it, (c) fix it, and (d) share it.

Figure 1 .
Figure 1.Proposed instructional maker education program framework.(Traditional engineering design steps were included)

Figure 2 .Figure 3 .
Figure 2. Short answer question in the programming skill test (students must describe the programming blocks in detail)

Figure 4 .
Figure 4.One student installing electronic sensors and gadgets onto the robot platform

Figure 5 .
Figure 5. Site plan diagram of the Robot MakerSpace

Figure 6 .Figure 7 .
Figure 6.Robot maze competition (the students programmed robots to independently travel through a Styrofoam maze)

Table 1 .
Experimental study design

Table 2 .
Research participant profile *Calculation by GpowerChou / Educational Robotics in Maker Education 6 / 15

Table 4 .
Results of the t test