Effects of the Multiple Solutions and Question Prompts on Generalization and Justification for Non-Routine Mathematical Problem Solving in a Computer Game Context

The aim of this study is to investigate the effects of solution methods and question prompts on generalization and justification of non-routine problem solving for Grade 9 students. The learning activities are based on the context of the frog jumping game. In addition, related computer tools were used to support generalization and justification of non-routine problem solving. Non-equivalent pretest/posttest quasi experimental design was adopted in this study. The experimental results are summarized as follows: (1) generalization and justification performance of non-routine problem solving for students in the specific prompt group is significantly better than that for students in the general prompt group, and (2) generalization and justification performance of non-routine problem solving for students in the multiple-solution group is significantly better than that for students in the single-solution group. Finally, suggestions were proposed based on the results found in this study, which may serve as useful directions for teachers and future studies.


INTRODUCTION
Generalization and justification have been known as the most important elements in algebraic activities (Blanton & Kaput, 2002).Dienes (1961) believed that generalization is an analogical process which takes certain numbers with similar properties and applies them to a broad range of numbers.For students, the analogical ability requires a long time to develop.If such analogical ability does not exist, algebra cannot be fully understood.To understand analogical process in algebra, students must see the algebraic rules and explain how they know these rules were true.Such algebraic rules can be viewed as mathematical expressions.Moreover, they can be regarded to represent any value under the defined variable domain.Therefore, justification becomes a crucial part of the analogical process.Blanton and Kaput (2002) believed that any form of justification is important in algebraic reasoning.In order to achieve generalization, learner will naturally adopt the mental habit of questioning and prediction.
In the study to investigate student's algebraic generalization and justification ability, it was found that students may encounter tremendous difficulty when they are identifying and establishing correct general statements and proofs (Knuth, Slaughter, Choppin & Sutherland, 2002).By examining student's behavior in algebraic pattern activities, it showed that even though students can identify various patterns, they cannot pay too much effort on algebraic activities relating to generalization (Blanton & Kaput, 2002).Studies showed that students tend to put their focus on pattern's recursive relation, rather than functional relation.This is the reason making generalization sometimes difficult (Blanton & Kaput, 2002).Besides that, effective identification of numerical patterns does not guarantee correct generalization for such patterns (Stacey & MacGregor, 1997).Even if students can generalize a pattern or rule, very few of them can explain the outcome.
Regarding to the difficulties encountered by students during generalization and justification, how to provide an appropriate teaching strategy for them to improve their performance in generalization and justification is an important topic to be studied.Previously, studies were mainly focused on describing the current status of generalization and justification performance.Few of them addressed the important issues toward the teaching strategy.The use of multiple solution method might be an effective approach.According to the Ainsworth's (2006) viewpoint of multiple representation, multiple solution method may help learners to improve their performance in generalization and justification through the following features: (1) the use of multiple solution can offer complementary information or support complementary cognition, (2) a solution may be used to limit the possibility of another solution (wrong solution), and (3) different solutions may encourage learners to think more and understand different problems in more detail.Nevertheless, such learning approach may be highly demanded for learners since they must integrate mentally different methods to obtain the representation consistent with the teaching materials.If the teaching materials is not well designed, cognitive load may appear.To prevent the occurrence of such problem, combination of multiple solution method with other learning strategies that may reduce cognitive load is reasonable.The use of worked example together with multiple solution method might be an effective way of learning.This is because using worked examples can reduce the cognitive load, allowing learners to have more cognition capacity for carrying out multiple solution integration (Grobe & Renkl, 2006).As a result, the use of multiple solution worked examples was adopted in this study for the design of teaching materials.In addition, the effects of adopting such learning strategy on generalization and justification performance for students were investigated.
Many researchers (Ge & Land, 2003;Schoenfeld, 1985) suggested that students should be encouraged to engage in non-routine mathematical problem solving.This not only helps students to understand more about the meaning of the acquired knowledge and the connection between them, but also facilitates the transfer of knowledge for students in real life scenario.In addition, computer game has been regarded as an effective mathematics learning tool because it can make the teaching materials highly attractive (Moreno, 2002).Many evidences supported the fact that (Bahr & Rieth, 1989;Inkpen, 1994) using computer games to teach students in fundamental mathematics and problem solving have positive effect.However, O'Neil, Wainess and Becker (2005) believed that using computer games alone is ineffective.They believed that computer game works only if other learning support is used simultaneously.
Therefore, the successful implementation of computer games in class depends on the quality of teaching, which includes the ability of teachers to reveal students' talents, ensure the learning targets and use appropriate ways to develop games meeting the learning objectives.

State of the literature
• Generalization and justification are known as the most important elements in algebraic activities.

91
It is well known that solving non-routine problem requires students with high-level inference skill and intensive learning support (Johnassen, 1997), such as demonstration, coaching and scaffolding.Prompting has been proved to be effective for scaffolding student's higher order thinking in various fields and circumstances (Scardamalia, Bereiter, McLean, Swallow & Woodruff, 1989).It can also encourage students to perform self-explaining (Chi, Lewis, Peimann & Glaser, 1989), self-questioning (King, 1991(King, , 1992)), and selfmonitoring/reflection (Lin, 2001).These activities can help learners to elaborate their thinking and to make inference.Most importantly, these activities can help learners to monitor and evaluate their own learning process.Prompting is consisted of procedural prompts, elaboration prompts, and reflection prompts.Each prompt is appropriate for different cognition and metacognition.Procedural prompt is designed to help learners completing specific mission, such as writing (Scardamalia, Bereiter & Steinbach, 1984) or problem solving (King, 1991).It has been successfully implemented to help learners understanding cognitive strategy in certain fields (Rosenshine, Meister & Chapman, 1996).Elaboration prompt, on the other hand, is designed to remind learners how to accurately express their thoughts and induce their explanation.Reflection prompt is designed to encourage students to perform reflection in metacognition level, which acts as complement for conditions that are not often considered (Davis & Linn, 2000).
Different prompts may offer different needs and goals for students.Recently, researchers have been trying hard to find out how different prompts can promote students' cognition and metacognition.For instance, Davis (2003) examined the productivity of reflection for science students through the use of generic prompts and directed prompts.Generic prompts only require students to stop and think.On the contrary, direct prompts are more elaborative, which provides the direction and strategy of reflection for students.Davis found that generic prompts can induce more reflection behaviors as comparing with direct prompts.Chen and Bradshaw (2007) examined the conceptual understanding and the performance of illstructured problem solving for student teachers through the use of knowledge integration prompts and problem solving prompts.Knowledge integration prompt is designed to guide students in concept connection, concept comparison, evidence finding, thought converting and correlation summary.Problem solving prompt, on the other hand, is designed to focus students' attention on specific procedure for problem solving.Chen and Bradshaw found that students who accept knowledge integration prompt tend to have higher scores in the overall problem solving performance as compared with students who accept problem solving prompt.Therefore, the results of examining different types of prompt are still inconsistent, and further investigation or clarification is still needed.
Most studies on non-routine mathematical problem solving are based on the analysis of small samples or specific target cases (Chen & Bradshaw, 2007;King, 1991;Schoenfeld, 1985), such as college/university students, graduate students and professors.Therefore, in this study, the focus will be shifted to the non-routine mathematical problem solving for junior high school students and the number of participants involved in the experiment will be increased.Even though Ge and Land (2003) pointed out that prompting has a positive effect on non-routine problem solving for students, they did not examine the effect of student's prior knowledge on non-routine problem solving, other than the self report based on different problem solving experiences.Moreover, Ge, Chen and Davis (2005) also suggested that future studies on student conceptual knowledge and non-routine problem solving performance should compare the effects generated by different types of prompts.As a result, this study used frog-jumping game as the context to present challenges for students.Consequently, the curiosity of students can be induced.The major two non-routine tasks relating to the frogjumping game were provided to students for learning.The purpose of these tasks is to promote student's performance in pattern reasoning and problem solving.The web-based learning environment was designed to provide students with the required information for mathematics learning.This allows students to perform real problem solving activities.In addition, through the use of various prompts, student's learning process can be guided.
In summary, the aim of this study is to investigate the effects of applying various multiple solution worked examples and prompting strategies for solving problems in computer game-based context on non-routine problem solving performance of generalization and justification.The major questions for this study are as follows: 1. Is there a significant interaction between different solution methods and different prompting strategies on students' generalization performance of non-routine problem solving after controlling the influence of prior knowledge?
2. Is there a significant interaction between different solution methods and different prompting strategies on students' justification performance of non-routine problem solving after controling the influence of prior knowledge?

Participants
Four classes of Grade 9 students in a junior high school in Taoyuan County were selected as the participants.Each of the four classes were assigned randomly as multiple solution-specific prompt group (31 students), single solution -specific prompt group (30 students), multiple solution-general prompt group (30 students), and single solution-general group (29 students).The effective samples were summed up to a total of 120 students as shown in Table 1.The teaching activities for the four groups were carried out by the same teacher.

Research Procedure
Before the teaching experimental activity in this study, pattern reasoning tests were performed.Then, multiple solution and single solution method were conducted separately for students learning of prior knowledge.After that, teaching for non-routine mathematical problem solving was carried out based on specific prompts and general prompts.Finally after teaching, the learning worksheets at this stage were collected.The learning worksheets were collected as the examination for the generalization and justification performance of non-routine mathematical problem solving.The results were reported based on the gathered data.
To answer question 1 and 2 of this study, two-way factorial (solution method: single solution vs. multiple solution, question prompt: specific prompt vs. general prompt) ANCOVAs (analysis of covariance) were adopted to investigate the differences on generalization and justification performance of students' non-routine mathematical problem solving.

Pattern Reasoning Test
Pattern reasoning test proposed by Lee and Chen (2009) was adopted to understand students' prior knowledge before solving the frog-jumping problem.
The questions in the test were decided based on the task analysis of the frog-jumping problem.In addition, the basic pattern reasoning ability needed to perform the frog-jumping problem was determined.The test questions can be divided into three parts: recursive relation, functional relation and algebraic expression.Each question is worth 5 points to give a total of 60 points.The points obtained in the pattern reasoning test can be regarded as the prior knowledge in this study.
After conducting the test, the internal consistency among questions was examined by Cronbach's α reliability analysis.A reliability value of 0.869 was obtained for the entire test, whereas the reliability value for each subscale of the test as mentioned previously was 0.712, 0.661, and 8.13, respectively.Therefore, the internal consistency of the test is acceptable.Furthermore, the test performed in this study was reviewed by two professors in related field as well as three rich-experienced mathematics teachers in junior high school.As a result, the expert content validity for this pattern reasoning test is quite good.

The design of website and learning worksheets
To help students understand more about the frogjumping problem (see Appendix 1), researchers have used Flash to develop computer-assisted learning tools for the mathematics teacher.These tools will help the teacher incorporate information technology in teaching and assist the teacher to design digital materials for teaching mathematics.The website provides many tools, prompts, and expert examples to help students completing the learning tasks.In addition, the learning worksheets were designed in accordance with the instructional website for frog-jumping problem.Students need to complete the non-routine tasks on the worksheets by following teacher's instruction and the supporting materials in the website.From this, reasoning and thinking related to students' non-routine problem solving can be collected.In this study, the generalization performance of non-routine problem solving can be evaluated according to

93
the other hand, the justification performance of nonroutine problem solving can be evaluated based on Table 3.A score of 0 (zero) means no justification.A score of 1 means appeal to external authority.A score of 2 means empirical evidence and a score of 3 means generic example.Therefore, the highest score of justification performance for non-routine problem solving is 3.
Frog-jumping website and learning worksheets were developed in three months.During the course of the development and revision period, classes of the same background as the study participants were selected to perform experimental teaching.Suggestions from students, teachers and class observers were collected for revision so that the teaching requirements can be fulfilled.In the end, expert validity was performed by several junior high school mathematics teachers and three professors in related field to complete the revision process.

Experimental Treatment
In task 1 and task 2, learners were divided into multiple-solution worked example group and singlesolution worked example group.There are at least 2 solutions in the examples provided to students in the multiple-solution group and students were required to analyze the advantages and disadvantages of multiplesolution method.In the single-solution group, only one solution was provided for each example.The solutions provided in the multiple-solution group were also available in the single-solution examples, only these solutions were found in different single-solution examples.Both groups have provided students with learning worksheets that contain procedures for selfexplanation.The worksheets can be used to record student's learning process and thinking strategy.In task 3 and task 4, learners were divided into specific prompt group and general prompt group.Three levels of prompts were offered by the specific prompt group, ranging from general process prompt to specific strategy prompt.By going through these prompts, learners will have a chance to elaborate their thinking process and have more opportunities for reflection.As for the general prompt group, prompts of expert's solution processes were provided to learners, hoping they can mimic expert's problem solving behaviors and thinking, hence, solve the problems successfully.General speaking, general prompt tends to provide procedural scaffolding that describes the general direction, whereas specific prompt not only provides generic prompt, but also specific strategic direction for learners.Specific prompt offers three levels of prompt, prompt 1, prompt 2 and prompt 3.These three levels of prompt range from general prompt to specific prompt, which provides more opportunities for learners to think and reflect, hence, come up with a useful problem-solving strategy (see Figure 1).Prompt 1 is for learners to observe the pattern of numbers.Prompt 2 is for learners to think the pattern of numbers using the circling method.Prompt 3, on the other hand, is for learners to find the pattern of numbers from the square and product of Present or express justification by using a specific example these numbers.As for general prompt, only one level of prompt, prompt 1, is available.Its purpose is to guide learners in finding pattern of numbers from the learning worksheets so that problem can be solved successfully (see Figure 2).Students must follow the instructions given by teachers and answer questions according to the prompts provided on the learning worksheets.Such operations will ensure students to fulfill both elaboration and refection.According to the two-stage grouping criteria mentioned previously, students can be divided into four groups, namely multiple-solutionspecific prompt group, single-solution-specific prompt group, multiple-solution-general prompt group, and single-solution-generic prompt group.Learning

95
worksheets were designed based on these four groups.In addition, frog-jumping problem instructional website was established to provide relevant teaching materials.
Every student must complete the learning worksheets in accordance with the tasks and prompts provided on the frog-jumping instructional website.The solution for each task will be provided upon completing the worksheets and before revealing the next task, meaning that teachers will not provide students solution to any question in the worksheet.Students will know the answer of each question after its corresponding worksheets have been collected.Therefore, evaluation of the worksheets can provide valuable information for student's problem solving process.

Is there a significant interaction between multiple solution and question prompt on generalization performance for non-routine problem solving?
The adjusted mean of generalization performance for non-routine problem solving in multiple-solution group (Adjusted Mean=1.70) is slightly higher than that in single-solution group (Adjusted Mean=1.43).Moreover, the adjusted mean of generalization performance for non-routine problem solving in specific prompt group (Adjusted Mean=1.62) is higher than that in general prompt group (Adjusted Mean=1.16).Since the covariance test on the homogeneity of regression coefficient within groups is not significant (F=.78, p=.38) for different solution methods and question prompts, there must be a common regression line suitable for ANCOVA (analysis of covariance).Therefore, the F-test in two-way ANCOVA can be used to examine the generalization performance (significant or non-significant) for nonroutine problem solving under different solution methods and different question prompts.From Table 4, it is noticed that after controlling the influence of prior knowledge, the interaction between solution method and question prompt in generalization performance for non-routine problem solving is not significant (F=1.11).By looking at the result of each factor, it is found that the main effect of solution method is significant (F=4.73,p=.03, η2=.04), suggesting that the generalization performance for non-routine problem solving in multiple-solution group is better than that in single-solution group.Moreover, the main effect of question prompt is also significant (F=11.90,p=.00, η2=.10), implying that the generalization performance for non-routine problem solving in specific prompt group is better than that in general prompt group.
It is believed that the better generalization performance for non-routine problem solving in multiple-solution group as compared to single-solution group is due to the fact that multiple-solution group can integrate and compare the advantages and disadvantages of different solution methods in non-routine problem solving.As a result, a higher level generalization strategy can be selected.On the other hand, the better generalization performance for non-routine problem solving in specific prompt group as compared to general prompt group is attributed to the more concrete and detailed prompts offered by specific prompt group comparing with general prompt group.As a result, learners will have access to more evidences and resources, which can then be integrated with their prior knowledge to figure out higher level generalization strategy.
The generalization strategy for non-routine problem solving in multiple-solution group and single-solution group were further investigated (see Table 5 and 6).The percentage of comprehensive strategy used by multiplesolution group (67%) is higher than that used by singlesolution group (38%), suggesting that multiple-solution worked examples have the benefit to improve the use of comprehensive strategy for students.
The generalization strategy for non-routine problem solving in specific prompt group and general prompt group were also investigated (see Table 7 and 8).The percentage of comprehensive strategy used by specific prompt group (61%) is higher than that used by general prompt group (44%).In addition the percentage of no strategy or wrong strategy encountered by specific prompt group is lower than that by general prompt group.This suggests that specific prompt has the benefit to improve the use of comprehensive strategy for students.Furthermore, it has the benefit to reduce the percentage of no strategy or wrong strategy.

Is there a significant interaction between multiple solution and question prompt on justification performance for non-routine problem solving?
The adjusted mean of justification performance for non-routine problem solving in multiple-solution group (Adjusted Mean=1.77) is higher than that in singlesolution group (Adjusted Mean=0.90).Moreover, the adjusted mean of justification performance for nonroutine problem solving in specific prompt group (Adjusted Mean=1.66) is higher than that in general prompt group (Adjusted Mean=1.02).Since the covariance test on the homogeneity of regression coefficient within groups is not significant (F=.54, p=.47) for different solution methods and different question prompts, there must be a common regression line suitable for ANCOVA (analysis of covariance).Therefore, the F-test in two-way ANCOVA can be used to examine the justification performance (significant or non-significant) for non-routine problem solving under different solution methods and different question prompts.From Table 9, it is noticed that after controlling the influence of prior knowledge, the interaction between solution method and question prompt in justification performance for non-routine problem solving is not significant (F=.22).By looking at the result of each factor, it is found that the main effect of solution method is significant (F=12.53,p=.00,As a result, a higher level justification strategy can be selected.On the other hand, the better justification performance for non-routine problem solving in specific prompt group as compared to general prompt group is attributed to the more specific prompts.Concrete and detailed prompts can provide students more opportunities to think about one problem from different perspectives.As a result, learners may have more resources, which can then be integrated with their prior knowledge to use more comprehensive generalization strategy and then to use higher level justification strategy.

CONCLUSION
Based on the results and discussion mentioned in the previous section, some important conclusions found in this study are summarized as follows: The role of question prompt in solving non-routine problems After controlling the influence of prior knowledge, students in specific prompt group showed better generalization and justification performance for nonroutine problem solving as compared to those in general prompt group.This suggests that for more difficult and complicated problems, specific prompt group offers more detailed prompts comparing with general prompt group, which benefits the performance of students in non-routine problem solving.In addition, specific prompt has the benefit to improve the use of comprehensive strategy for students.As a result, the percentage of no strategy or wrong strategy can be reduced.

The role of multiple solution in solving non-routine problems
After controlling the influence of prior knowledge, students in multiple-solution group showed better generalization and justification performance for nonroutine problem solving as compared to those in singlesolution group.It is believed that the better performance for non-routine problem solving in multiple-solution group as compared to single-solution group is due to the fact that multiple-solution group can help students to integrate different problem-solving strategies.In addition, multiple-solution worked example has the benefit to improve the use of comprehensive strategy for students in solving nonroutine problems.

SUGGESTIONS
Based on the above findings and conclusions, the following suggestions are proposed.

Suggestions for junior high school teachers in teaching problem solving
Since specific prompt group showed better performance for non-routine problem solving as compared to general prompt group, teachers when designing teaching programs for non-routine problem solving should provide more concrete and detailed specific prompts for students, so that students will have the chance to integrate their own knowledge and apply it to the process of non-routine problem solving.When expert's problem-solving prompt cannot successfully connect with the knowledge or state of students required for solving problems, more specific or detailed prompt is needed.Therefore, the design of prompt for each task, from general to specific, is particularly important.Specific prompt offers more substantial teaching efforts to connect with student's prior knowledge and skills for non-routine problem solving.
Since multiple-solution group showed better generalization and justification performance for nonroutine problem solving as compared to single-solution group, teachers should provide multiple-solution worked examples to students to improve the level of students' generalization and justification skills in nonroutine problem solving.By doing so, students will have more chances to integrate their resources, which in turn improves their level of generalization and justification skill in non-routine problem solving.
Students usually use different strategies or justification to construct or explain the same generalization task.Therefore, instructional design for generalization should try to induce student's interest in using different generalization strategies and discuss the validity of different justifications.In the future, the instructional design may encourage students to check other students' justification and generalization strategy and to examine the impact of these activities.Multiple-solution worked example will have the benefit to increase the usage of comprehensive strategy and reduce the usage of partial strategy in non-routine problem solving for students.In contrary, singlesolution worked example will not have the benefit to increase the usage of comprehensive strategy and reduce the usage of partial strategy in non-routine problem solving for students.Therefore, teachers should use multiple-solution worked examples in their teaching design for non-routine problem solving.In this case, students can solve the problem from different perspectives.Furthermore, students will more likely to use comprehensive strategy in their non-routine problem solving.
Specific prompt has the benefit to increase the usage of comprehensive strategy and reduce the usage of partial strategy in non-routine problem solving for students.In contrary, general prompt has no benefit to increase the usage of comprehensive strategy and reduce the usage of partial strategy in non-routine problem solving for students.Therefore, teachers should use specific prompt in their instructional design for nonroutine problem solving.In this case, students will have more opportunities and evidences to think and solve problems.Furthermore, students will more likely to use comprehensive strategy in their non-routine problem solving.

Suggestions for future research
In this study, the sample size is small.Therefore, the results and conclusions obtained in this study are somewhat limited.It is hoped that in the future, more participants can be selected to further study the effect of solution method and question prompt on non-routine problem solving.This study focuses only on pattern reasoning.We hope that in the future, the effect of multiple-solution and specific prompt on non-routine problem solving in different knowledge fields, such as biology or chemistry, can be investigated further.The difficulty and characteristics of each learning task will influence the effect of multiple solution and specific prompt.Therefore, a systematic study on the characteristics of different learning tasks may be conducted to clarify this issue.
Different forms of multiple-solution worked examples (such as single representation multiplesolution method or multiple representation multiplesolution method) may generate different effects on the learning tasks.In addition, characteristics of the solution method itself, such as formal or informal solution method, may also generate different effect on student's learning.This part can be further investigated in the future.Our project has not been incorporated into school's regular teaching program.Only one additional class in each week was used to implement this project.As a result, the teaching and learning effect observed here can not be extended entirely to the actual situation.We hope that the frog-jumping problem project can be incorporated into school's regular teaching program, so that its real effects can be revealed.
The misconception of non-routine generalization for students was not discussed in this study.Therefore, qualitative research method can be used to deeply examine students' misconception on non-routine problem solving in the future.Finally, the participants can be divided into partial strategy group and comprehensive strategy group.The performance of these two groups in non-routine problem solving can be further investigated.The interaction between different strategies (such as multiple solution or specific prompt) and different generalization strategies (partial or comprehensive) on students' learning performance can be examined in depth.

Figure 1 .
Figure 1.Design of specific prompt

Figure 2 .
Figure 2. Design of general prompt

Table 1 .
Summary of samples selected in this study Note: a(b,c) where a represents the number of students in that group, b represents the number of boys in that group, and c represents the number of girls in that group.

Table 2 .
Quadratic relation generalization strategy

Table 4 .
Summary of two-way ANCOVA for generalization performance of non-routine problem solving =.11), suggesting that the justification performance

Table 5 .
Generalization strategy for non-routine problem solving in multiple-solution group

Table 6 .
Generalization strategy for non-routine problem solving in single-solution group

Table 7 .
Generalization strategy for non-routine problem solving in specific prompt group

Table 8 .
Generalization strategy for non-routine problem solving in general prompt group

Table 9 .
Summary of two-way ANCOVA for justification performance of non-routine problem solving