Using the TOSRA to Assess High School Students ’ Attitudes toward Science after Competing In the FIRST Robotics Competition : An Exploratory Study

This study examined high school students’ attitudes toward science after participating in a robotics competition. Specifically, this study used the Test of Science Related Attitudes (TOSRA) to measure students’ attitudes toward science in seven categories: Social Implications of Science, Normality of Scientists, Attitude toward Scientific Inquiry, Adoption of Scientific Attitudes, Enjoyment of Science Lessons, Leisure Interest in Science, and Career Interest in Science. Results indicate that students who participated in the robotic competition had a more positive attitude toward science and science related areas in four of the seven categories examined: Social Implications of Science, Normality of Scientists, Attitude toward Scientific Inquiry, and Adoption of Scientific Attitudes. Implications of results on students’ attitudes are discussed.


INTRODUCTION
One of the key challenges facing the field of science education is recruiting, educating, and retaining students in the field of the sciences, technology, engineering, and mathematics.Within the next decade, "the number of individuals with science and engineering degrees reaching typical retirement age is expected to triple" (National Science Foundation, Science & Engineering Indicators -2002, p. 31).In 1999, among 3,540,800 persons employed in science and engineering occupations, only 1,032,100 had Master degrees and 484,100 had earned Doctorate degrees (Wilkinson,p. 2).In a report from the Merrill Advanced Studies Center, Ortega states, "…the fundamental problem is the declining percentage of students in science, technology, engineering, and mathematics (STEM) graduate programs, especially at the doctoral level" (Ortega, 2003).In addition, student interest in science, mathematics, and engineering fields continues to be a concern.
A national study, examining trends in undergraduate education, reveal a steady decline in student interest in the physical sciences and mathematics over the last thirty years (Astin, 1997).Female, African-American, and Hispanic students appear to have lower level of interest in the sciences than do male, Asian and Caucasian students (National Science Board, 2002).Therefore, the challenge today is two-fold.First, we must successfully prepare students for careers in science and mathematics, and second, we must increase students' interest in science, mathematics, engineering, and technology, especially students from diverse backgrounds.
The robotics competition examined in this study, the FIRST Robotics Competition (FRC), was designed to increase high school students' interest in science, technology, engineering, and mathematics (STEM) and to increase interest in pursuing STEM related fields.
The key component of the robotics competition is a six-week period where teams of high school students design and build a robot designed for a specific task.Teams composed of between twelve and twenty students worked with a high school teacher and mentors from local universities, professional organization, and/or businesses to build a robot they can use to compete against other teams from across the United States.
In this study, a convenience sample of high school students who participated in FRC was compared to a convenience sample of students from the same schools who did not participate in the program.Students' attitudes were measured both before and after the sixweek design and build period in order to examine the impact of participation in the robotics competition on their attitudes toward science and science related issues.The study was designed to answer the following research question: Does participation in a robotics competition change high school students' attitudes toward science and science related issues, as measured by the TOSRA (Test of Science Related Attitudes)?

Background Literature
The robotics competition examined in this study, the FIRST Robotics Competition (FRC), is designed to build awareness and interest in science and engineering in high school students by providing challenging and engaging learning opportunities in a setting which inspires students to pursue careers in science and technology in the same way professional sports inspires young people to pursue careers as professional athletes.
There are empirical studies have shown that projectbased learning environments, like FRC, can have a positive impact on student achievement in science and mathematics.A three-year study conducted in British secondary schools found significant differences in both student understanding and academic achievement in mathematics based on standardized test scores because of their participation in project-based schools (Boaler, 1999).The study found that three times as many students enrolled in the project-based schools earned the highest possible grade on the national examination in mathematics (Boaler, 1999).A similar study found that after project-based learning activities were used, eight graders in Union City, New Jersey, scored twentyseven percentage points higher than students from other urban school districts on statewide tests in reading, math, and writing achievement (Honey & Henriquez, 1996).This is significant given that four years prior to the implementation of project-based learning activities, the state had considered a takeover of the school because it had failed forty of fifty-two indicators of school effectiveness (Honey & Henriquez, 1996).

TOSRA
The TOSRA (Test of Science Related Attitudes) was used to assess changes in students' attitudes toward science and science related issues because of participating in FRC.Fraser (1978) (1971).In his classification system, Klopfer's first scale was called "Manifestation of favorable attitudes towards science and scientists" and was based on the following premises: …it is reasonable to see whether the student will speak, write, and act in ways which show that he places a positive value on the role of science in furthering man's understanding and that he give due acknowledgement to scientists for their past and potential future contributions in their quest.(p.577)

State of the literature
 Previous empirical studies have shown that project-based activities can have a positive impact on student achievement in both science and mathematics. In addition, important factors to success in the classroom are motivation and attitude. Students who are authentically engaged in the learning process make meaningful connections to real-world situations.

Contribution of this paper to the literature
 This is the first study to use the TOSRA to study students who participated in the FIRST Robotics Competition. This study implies that programs that engage students in authentic science related learning activities can significantly improve students' attitudes and views of science. There is a correlate between positive views toward science and achievement in science.As a result, programs, such as the one studied here, may help improve students overall achievement in science.
Fraser divided Klopfer's original classification into the Social Implications of Science scale and the Normality of Scientists scale.
The Social Implications of Science scale measures the "manifestation of favorable attitudes towards science" (Fraser, 1981, p. 2).This includes attitude towards the social benefits and problems associated with scientific progress and research.
The Normality of Scientists scale measures the attitude toward scientists as normal people rather than eccentrics.This scale measures how students perceive scientists as individuals and their perceptions of scientists as having a normal lifestyle.
The Attitude of Scientific Inquiry scale measures attitude toward scientific experimentation and inquiry as methods of obtaining information about the natural world.This scale measures the acceptance of scientific inquiry as a way of thought.Fraser maintained Klopfer's classification based on the idea that "…if a student accepts the processes of scientific inquiry as a valid way to conduct his thinking, his behavior in approaching a problem or novel situation will be sufficiently consistent for competent observers of his action to describe him as behaving just like a scientist" (Klopfer, 1971, p. 577).
The Adoption of Scientific Attitudes scales measures open-mindedness, willingness to reverse opinions related to scientific investigation and inquiry.This scale measures the how likely students are to change their way of seeing the world based on scientific evidence.Fraser maintained Klopfer's classification of this scale.
The Enjoyment of Science Lessons scale measures the enjoyment of science learning experiences.This includes participating in science labs as well as attending science classes.According to Klopfer, "…the sight, sound, and smell of phenomena; the uncovering of a new relationship, generalization, or explanation the spark of discussions of conflicting ideas -these are all potential sources of involvement and enjoyment" (p.578).
The Leisure Interest in Science scale measures the development of interest in science and science-related activities.Fraser's category is designed to reflect the students' interests in hobbies and extra-curricular activities outside of classroom related to science.This is a modification from Klopfer's original classification, which incorporated two aspects (Klopfer, 1971, p. 578):  The student's interests in activities that he can carry out himself (voluntary participation). The attention he gives to the ongoing events in science and in the societal interactions of science. The Career Interest in Science scale measures the development of interest in pursuing a career in science (Fraser, 1981).Klopfer believed that the development of a student's interest in pursuing a career in science was "…a legitimate and worthy part of his learning in science" (p.578).Fraser's category follows much of Klopfer's original design.Each scale Fraser designed contains ten items; the total instrument contains 70 items.The response scale is a five point Likert scale with responses ranging from Strongly Agree (1) to Strongly Disagree (5).Within each scale, five are positive items, five are negative, with respect to their position on science, and science related issues.For the sake of clarity, all items have been adjusted so that a higher mean score is indicative of a more positive view of science.
After administering the survey to secondary students, Fraser calculated the reliability and validity of the survey.The values of the α reliability coefficient ranged from "0.66 to 0.93 with a mean of 0.82 for the Year 7 sample, from 0.64 to 0.93 with a mean of 0.81 for the Year 9 sample, and from 0.67 to 0.93 with a mean of 0.84 for the Year 10 sample" (Fraser, 1981, p. 4).The inter-correlations of the TOSRA scales were calculated as indices of discriminate validity.The intercorrelation was low and ranged from 0.10 to 0.59 with a mean of 0.33 (Fraser, 1981).
Fraser's research on the TOSRA was conducted with students in Australia.In 1987, a study was conducted to investigate the cross-cultural validity of the TOSRA when used with American high school students (Khalili, 1987).In the study, three hundred and thirty-six 11th and 12th grade students in suburb area Chicago high schools took the test.The researcher showed that the TOSRA did have a high degree of internal consistency when used with American students (Khalili, 1987).

Theoretical Background
Through the use of hands-on, real-life, problemsolving challenges, robotic competitions embody the ideals of constructivism and project-based learning.Theoretically, the FRC is based on a constructivist learning as conceptualized by Jean Piaget and social views of learning as conceptualized by Lev Vygotsky.According to Piaget, students possess an innate need to understand how the world operates and to find order, structure, and predictability in their existence (Piaget, 1952(Piaget, , 1959;;Eggen & Kauchak, 2001).According to Piaget, students are motivated by a need to understand the world and use adaptive schemes of assimilation and accommodation to organize knowledge into schemes.Experience with the physical world is critical to the formation of schemes and is found in most classrooms in the form of "hands-on" activities (Ball, 1992;Hartnett & Gelman, 1998).Piaget also emphasized the role of social experience in the learner (DeVries, 1997).It is critical that the learners be allowed to test their findings against those of others.This serves as a balancing effect and motivates the learners to adapt new schemes and compare views with those of others (DeVries, 1997;Eggen & Kauchak, 2001).
While Piaget examined the impact of experience, Vygotsky theorized that participation in social activities was vital to learning (Eggen & Kauchak, 2001;Bredo, 1997;Vygotsky, 1978).Vygotsky believed that learning occurs when students gain specific understanding and development progresses when this understanding is incorporated into a larger, more complex social context (Vygotsky, 1978).The FIRST Robotics Competition is designed to capitalize on the constructivist and social aspects of learning by providing students a real-world, problem they must solve as a team.
Dethlefs found that the constructivist learning environment dimensions of Personal Relevance, Shared Control, and Student Negotiation were positively related to student attitudes (Dethlefs, 2002).Empirical studies provide evidence that constructivist-learning environments in science and mathematics can have a positive impact on students' attitudes in both science and mathematics.In a study conducted with sixty-two high school students, Nichols and Miller found that those students assigned to constructivists learning groups showed that greater gains were made in achievement, efficacy, valuing of algebra, and learning goal orientation (Nichols & Miller, 1994).Shymansky, Hedges and Woodworth confirmed earlier meta-analysis studies which supported findings that student performance was increased through the use of inquirybased science curricula dating back to the 1960s (Shymansky, Hedges & Woodworth, 1990).

Student Attitudes in Science
Constructivism and project-based learning are also an important factor in student attitudes and motivation.Motivation is considered one of the most significant determinants of students' success or failure in the classroom (Hidi & Harackiewicz, 2000;Reeve, 1996;Ryan & Connell, 1989).Studies have shown that active involvement in learning activities is more motivating than being passive involvement (Zahorik, 1996).In addition, student control and responsibility are also associated with increased motivation, which translates into increased learning and retention of information (Lepper & Hodell, 1989;Eggen & Kauchak, 2001).
In the 1990s, studies were conducted using select groups of students, such as at-risk, urban, or those with various disabilities.One such study highlighted the difficulties of engaging African American students who live in relative poverty when they lack motivation to learn and attend class sporadically (Tobin, et al., 1999).Motivation was also the subject of a study by Dicintio and Gee (1999).They found that their test group of atrisk students was "unmotivated to learn, defiant in learning situations, and evidence a negative attitude toward school (p.234).They concluded that "educators and researchers working with at-risk students should be encouraged to try current, innovative, cognitively based methods of motivating students…[and that] at-risk students need to learn the skills of self-determination and adaptive motivation in school learningcharacteristics that cannot be imparted through motivational practices that control and coerce students" (Dicintio & Gee, 1999, p. 235).
The concept and definition of attitude has been extensively studied (Eiser, 1984;Lemon, 1973;Mueller, 1986;Thurstone & Chavez, 1929).Attitude has been described as a non-observable psychological entity, which can only be deduced from a manifested behavior (Adolpe, 2002;Mueller, 1986).Thurstone initially described attitude as "the sum total of a man's inclination and feelings, prejudices and bias, preconceived notions, ideas, fears, threats, and conviction about any specified topic" (Thurstone, 1928, p. 531).He later modified his definition stating attitude was the "effect for or against a psychological object" (Thurstone, 1931, p. 261).He recanted this definition in a 1946 commentary, stating he actually believed that attitude was more accurately described in his earlier work as "the intensity of positive or negative effect for or against a psychological object" (Thurstone, 1946, p. 39).Additional definitions of attitude include "a mental or neural state of readiness" (Allport & Hartman, 1935, p. 810), a "consistency in response to social objects" (Campbell, 1950, p. 31), and "the covert response evoked by a value" (Linton, 1945, pp. 111-112).
The term attitude encompasses a wide variety of affective behaviors, such as prefer, accept, appreciate, and commit.In most studies, the term "attitudes" is used to refer to the intrinsic values or interests of the students toward science and mathematics (Dethlefs, 2002).In 2000, Dethlefs conducted a study on the relationship of constructivist learning to students' attitudes and achievement in high school science and mathematics.His findings showed the following results:  Constructivist learning environments are positively associated with student attitudes in high school biology and algebra. Deeper cognitive processing strategies were present when students were allowed to exercise more control in their learning activities. Students' enrollment in future elective classes was predicted as a result of their attitudes. There is a strong relationship between cooperative group-work and students' interest in school.Studies have examined the relationship of attitude and achievement in the sciences.Student attitude toward science has been shown to correlate with achievement in the science classroom (Germann, 1988;Napier & Riley, 1985).In 1986, Schibeci & Riley studied the relationship between students' background, perceptions, attitudes, and achievement.Their study showed that gender is related to attitudes and achievement, with females scoring lower on both.In 1990, Hill, Pettus, and Hedin found a lack of interest in science careers and lack of participation in science related activities, outside of school, with middle and high school girls.
Research has been conducted to investigate the relationship of individual interest to situational interest (Dethlefs, 2002).Situational interest has been shown to enhance students' individual interest (Mitchell & Gilson, 1997).Situational interest manifests itself in the classroom as authentic engagement.Students who are authentically engaged experience understanding in what they are doing are able to make meaningful connections.These students may participate in the tasks because they can see a link between what is being done and the significance of the outcomes of their work.Student engagement relates to the meaningfulness of the activities, rather than the time and physicals effort expended on it.It is critical to focus on the meaning of the work rather than the amount of "activity" involved (Schlechty, 2002).

Study Participants
This study was conducted in a large Midwestern metropolitan area of the United States of America.Students in the study attended high schools, which were located in suburban, urban, and rural areas and included both public and private affiliations.The students in this study were from nine different high schools.All students who were members of the schools' FRC team were invited to participate in the study.The students who agreed to participate in the study comprised the treatment group.A comparison sample of students was comprised of students from the same schools, who were not on the FRC teams, but who were enrolled in the same science classes as the students on the robotics teams.
The treatment group (FRC) consisted of 80 participants in the pre-survey and 58 in the post-survey.The comparison group (non-FRC team members) consisted of 52 in the pre-survey and 41 in the postsurvey.The gender of the students completing the presurvey was calculated at 36.3% female and 63.8% male in the FRC team members group and 51.9% female and 48.1% male in the non-FRC team members group.The gender of the students completing the post-survey was calculated at 37.3% female and 62.7% male in the FRC team members group and 52.5% female and 47.5% male in the non-FRC team members group.The race profile of FRC team members completing the post-survey was calculated at 81.4% white and 18.6% non-white.The race profile of non-FRC team members completing the post-survey was calculated at 62.5% white and 37.5% non-white.
A total of 132 students completed the pre-survey.Eighty students reported that they were members of FRC teams; fifty-two reported they were not members of a FRC team.Ninety-nine students who completed the pre-survey also completed the post-survey.Fiftyeight reported they were members of FRC teams; fortyone reported they were not members of a FRC team.Students completing the post-survey represented a 74.43% participation rate.
Inferential statistics using the post-survey results indicated the two groups were comparable.The results of a Chi-square test were not significant for females, χ2 (1, N =56) = .07,p < .01,but were significant for males, χ2 (1, N = 76) = 8.90, p < .01.These results show that the two groups were statistically similar concerning the number of female students in the sample, but were not concerning the number of male students.The results of a Chi-square test were significant for white students, χ2 (1, N = 94) = 13.79,p < .01,however, not significant for non-white students, χ2 (1, N = 38) = 1.68, p < .01.These results show that the two groups were statistically similar concerning the number of non-white students, but were not similar concerning white students.
An independent-samples t test was conducted on FRC members and non-FRC team members groups overall GPA in science classes to evaluate whether their mean was significantly different.
An independent-samples t test was conducted on FRC team members and non-FRC team members groups overall GPA to evaluate whether their mean was significantly different.The test was not significant, t(118) = .182,p = .12.The students participating in FRC (M = 2.44, SD = .86)reported approximately the same GPA as students not participating in FRC (M = 2.41, SD = .81).Therefore, the two groups appear statistically similar.
Overall, the results indicate that the two groups appear comparable.There were only two differences between the two groups.The results indicate that the treatment group had slightly more male students but had a slightly lower GPA in science classes.

Instrumentation
Each scale on TOSRA contains ten items, while the total instrument contains 70 items.A sample of questions from the TOSRA in listed in the Appendix.The response scale is a five point Likert scale with responses ranging from Strongly Agree (1) to Strongly Disagree (5).Within each scale, five are positive items, five are negative, with respect to their position on science, and science related issues.All items were adjusted so that the higher numeric values associated with the response categories of the items always reflected the positive side of the Likert scale indicating a more positive view of science.The coefficient alpha of .97suggests that the TOSRA was reasonably reliable for respondents in this study.

Research Time Period
Pre-surveys were administered prior to the six-week build season.
Post-surveys were administered immediately following the conclusion of the six-week build season.All surveys were in administered in faceto-face format at the high schools.

Statistical Methods
ANCOVA was used to determine whether the means of the dependent variable for two or more groups of the independent variable differed significantly when the influence of another variable that was correlated with the dependent variable was controlled.The level of significance for each variable, the p-value or probability, was analyzed for the established alpha level of 0.05.For this study, the p-values less than 0.05 were considered statistically significant, which represents the critical value most commonly used in behavioral statistics.

Social Implications of Science
A preliminary analysis evaluating the homogeneityof-slopes assumption indicated that the relationship between the covariate and the dependent variable did not differ significantly as a function of the independent variable, F(1, 95) = 1.53,MSE = 28.52,p = .22,partial 2 = .02.Therefore, the homogeneity of slopes test indicated that the assumption had been met and that the results of an ANCOVA would be meaningful.
A comparison of the pre-and post-means indicated that students who participated in FRC had a more positive attitude toward the social implications of science than students who did not, as shown in Table 1.
The FRC participants recorded a 0.64 increase in the mean difference whereas the non-FRC participants recorded a -0.93 mean difference.Students in the FRC group had statistically significant higher attitude means, (p < 0.01), than students in the comparison group regarding their attitude about the social implications of science as measured by the TOSRA.The strength of the relationship between the FRC factor and the dependent variable was small, as assessed by the partial η2 = 0.08, with the FRC factor accounting for 8.0% of the variance of the dependent variable.This indicates that the students participating in FRC had a greater appreciation for and more positive attitude of the importance of science and the social implications of science related issues than the non-FRC students due to their participation in FRC.

Normality of Scientists
A preliminary analysis evaluation the homogeneityof-slopes assumption indicated that the relationship between the covariate and the dependent variable did not differ significantly as a function of the independent variable, F(1, 95) = .43,MSE = 22.60, p = .51,partial 2 = .01.Therefore, the homogeneity of slopes test indicated that the assumption had been met and that the results of an ANCOVA would be meaningful.
A comparison of the pre-and post-means indicated that students who participated in FRC had a more positive attitude toward the normality of scientists than students who did not participate in FRC, as shown in Table 2.
Students in the FRC group had statistically significantly higher attitude means, (p = 0.011), than students in the comparison group regarding their attitude about the normality of scientists as measured by the TOSRA.The strength of relationship between the FRC factor and the dependent variable was assessed by a partial η2 = 0.065, with the FRC factor accounting for 6.5% of the variance of the dependent variable.
This indicates that students participating in FRC have a greater appreciation and more positive attitude towards scientists and members of the scientific community in general than students not participating in FRC.During the same time, the non-FRC students recorded a decrease in the mean.The increase in the mean of the FRC students may be a result of their interaction with technical mentors and engineers.

Attitude to Scientific Inquiry
A preliminary analysis evaluation the homogeneityof-slopes assumption indicated that the relationship between the covariate and the dependent variable did not differ significantly as a function of the independent variable, F(1, 95) = 1.25, MSE = 40.92,p = .27,partial 2 = .01.Therefore, the homogeneity of slopes test indicated that the assumption had been met and that the results of an ANCOVA would be meaningful.
A comparison of the pre-and post-means indicated that students who participated in the FRC had a more positive attitude toward scientific inquiry than students who did not participate in FRC, as shown in Table 3.
Students in the FRC group had statistically significantly higher attitude means, (p = 0.02), than students in the comparison group regarding their attitude toward scientific experimentation and inquiry as ways of obtaining information about the natural world and their acceptance of scientific inquiry as a way of thought, as measured by the TOSRA.The strength of relationship between the FRC factor and the dependent variable was small, as assessed by a partial η2 = 0.058, with the FRC factor accounting for 5.8% of the variance of the dependent variable.

Adoption of Scientific Attitudes
A preliminary analysis evaluation the homogeneityof-slopes assumption indicated that the relationship between the covariate and the dependent variable did not differ significantly as a function of the independent variable, F(1, 95) = 2.30, MSE = 28.85,p = .13,partial 2 = .02.Therefore, the homogeneity of slopes test indicated that the assumption had been met and that the results of an ANCOVA would be meaningful.
A comparison of the pre-and post-means indicated that students who participated in the FRC had a more positive attitude toward the adoption of scientific attitudes than students who did not participate in FRC, as shown in Table 4.
Students in the FRC group had statistically significantly higher attitude means, (p <0.05), than students in the comparison group regarding their openmindedness and willingness to revise opinions toward scientific experimentation and inquiry as measured by the TOSRA.The strength of relationship between the FRC factor and the dependent variable was small, as assessed by a partial η2 = 0.094, with the FRC factor accounting for 9.4% of the variance of the dependent variable.

DISCUSSION
The more favorable attitude of FRC students towards the social implications of science may be the result of several factors.While participating in the sixweek build period, students work directly at designing, building, and testing their team's robot.For many team members, this is the first time outside of a classroom setting, that they have had the opportunity to experiment and actually apply skills learned in the classroom.It is during this time that science becomes "real" and not just something found in textbooks.
The results of this study indicate that participants in FRC view scientist as more "normal" than students who do not participate in FRC, yet the FRC students' view of normal includes descriptions such as "geek."This indicates not only a shift in viewpoint toward scientists, but also a shift in the vernacular among FRC participants.While the general public often uses "geek" in a pejorative context, FRC students have adopted the term "geek" to represent a position of status and honor, of which they strive to be a part.Some of the explanation can also be found in the working relationship that is developed during the build season with the technical mentor, many of whom are engineers, who volunteer to work with the teams.For many students, this is their first interaction with scientists and engineers and the close relationship that is built influences their perceptions of the profession.Many students who participate in FRC discover that the terms "geek" and "nerd" do not apply to these individuals; they perceive them as "cool" and "hip." The Oxford English Dictionary (OED) defines "geek" as "a simpleton, a dupe, a person who is socially inept or boringly conventional or studious."The Urban Dictionary agrees with the OED definition of "geek" but also adds that in modern usage, the term "enjoys special status within the technical community," and that it "…indicates a recognition that most people still consider programming computers to be a bizarre act, along a certain fierce satisfaction in being very good at their inglorious profession."The term "geek" often carries a positive connotation when used by a member of the group.
Perhaps the explanation for the increase in attitude towards scientific inquiry by students participating in the FRC relates to the notion that the competition is built around the foundation of experimentation and inquiry.The focus of the FRC is the six-week build season in which teams have six weeks to complete the task of designing, building, testing, and shipping their robots.The six-week build season begins with the "Kick-Off."On the first Saturday of January, the "challenge" is presented via a simultaneous NASA broadcast from FRC in New Hampshire.It is during this broadcast that students find out specifics about the challenge and the rules, and are introduced to that year's kit of parts (the basic building components all teams start with for the competition).Following the broadcasts, teams meet to begin the brainstorming process and to gather the necessary information to best approach their solution to the challenge.The following weeks are filled with design meetings, building prototypes, construction of the robot, testing, redesigning, and finally shipment of the finished robot to a regional event.It is during these six weeks that the students work as much as possible, often well into the night and weekends.This is when the students actually apply the science and mathematics they learn in the classroom and when they work most closely with the technical mentors.
The ability to be open-minded and willing to reverse opinions is vital to members of FRC teams.During the design process, teams usually develop multiple plans and must select the best one based on their ability to build it and their available resources.This is often the most challenging time for the students, as each wants their design to be selected.Students must learn to negotiate and be willing to accept the ideas of other team members.An example of this was found with a local FRC team who had to make a formal presentation of their design to a doctoral candidate who works in biped robotics and served as the team's technical mentor.During this presentation, which also included other teachers and parents, the students had to answer questions from their mentor regarding various issues of force, torque, power, and other engineering principles.The students had to negotiate their opinions with his expertise.At the end of the evening, both were satisfied and the team proceeded with a slightly modified design, which went on to earn "Highest Rookie Seed" at a regional competition.This would not have been possible if the students had not been willing to alter their original plans and listen to the technical mentors.
While the previous studies on the impact of the FIRST Robotics Competition did not use the TOSRA, they do show similar outcomes as a result of student participation in the program.
In 1998, Atlantic Associates reported on students' attitudes and skills as affected by FRC.Their survey included not only the students participating in FRC, but also included the parents of the students, school personnel, and cooperate partners involved with the teams.The results showed that all vested parties in the program saw positive impact on students' problem solving ability, teamwork skills, self-confidence, and attitudes toward careers in engineering (Atlantic Associates, 1998).
The Goodman Research Group conducted a survey of the FIRST teams during the 2000 FRC season.This survey used pre-and post-data with 3,123 students participating from 150 FRC teams responding to the pre-survey and 400 responding to the post-survey.Even though the response rate was only 13%, the evaluation did provide a foundation for understanding the background of student participants, attitudes, and interest in science and mathematics, and interest in pursuing engineering careers (White Mountain Research Associates, 2002).According to the results of the survey, FRC attracts boys and girls of all different grade levels with each group having different levels of interest and commitment to science, technology, engineering, and mathematics.While the results showed little change between most responses, it did show that "students decided to participate in FRC mostly for academic reasons.They sought a challenging and educational science and mathematics experience and they expected that the experience would help them get into a better college" (Goodman Research Group, 2000, p. 52).Preand post-survey analysis also showed statistically significant increases in participants' attitudes toward teamwork and positive self-image (Goodman Research Group, 2000).
In 2001, White Mountain Research Associates reported that FRC had strong potential to influence the career choices of students.It also showed that interest in mathematics and science was strengthened because of the partnerships with various universities and sponsors (White Mountain Research Associates, 2001).
Brandeis University conducted a longitudinal study of the impact of FRC on participants.The final report, More than Robots: An Evaluation of the FIRST Robotics Competition Participant and Institutional Impacts, was published in April 2005.The major finding of the study reveals that the FIRST Robotics Competition "does appear to be successful in meeting the goals of promoting a positive academic trajectory for its students and a sustaining interest in science and technology-related education and careers" (Melchior, Cohen, Cutter, & Leavitt, 2005, p. 57).Nearly 90% of the alumni of the program attended college, a rate substantially above the national average.Once in college, FRC alumni were much more likely than nonparticipants to pursue courses and careers in science and technology-related fields.Forty-one percent of FRC participants listed engineering as their primary major, a number seven times the national average (Melchior, Cohen, Cutter, & Leavitt, 2005).FRC alumni were also more likely to attend college full-time, to have an internship, or coop job in their first year of college, and expected to attain some form of post-graduate degree.The study notes that while it cannot control for the initial motivation of the FRC students, the degree to which they were already interested in science and technology, the use of the matched comparison group of students with similar background in science in high school lends credence to the conclusion that FRC did make a difference in students' choice of college careers and that, without FRC, they would have been less likely to go into a science or technology-related field.Although the study showed that the overall impact on individual participants was strong, the impact of FRC on local schools was more modest.The participation in FRC did help some schools to introduce new courses, such as robotics, and increased school spirit.The final report notes that if great school impacts are desired, then a more "deliberate, school-focused strategy may be needed" (Melchior, Cohen, Cutter, & Leavitt, 2005, p. 58).
This study has important implications for the field of science education.The results imply that programs that engage students in authentic scientific problems can significantly improve students' attitudes and views of science.The FRC program appears to be helping students develop a more positive attitude and interest in science, as shown by the TOSRA.Positive views toward science are often viewed as an important correlate to achievement in science, and as a result, programs like FRC that can help improve students' attitudes and interest in science may be an important part of helping students achieve in science.The positive attitudes and interest may also lead to future careers in science related fields, but more research will be needed to better understand the long-term impact of the program on the students.