Preservice Science Teachers ’ Instructional Design Competence : Characteristics and Correlations

Underpinned by a holistic, dynamic, and process-oriented view of teacher competences, this study provides an analytic hierarchy system of instructional design competence (IDC) for evaluating teachers’ IDC based on the mental model of instructional design. Additionally, this study quantitatively explores the IDC characteristics and correlations of 118 preservice science teachers at Shanxi Normal University in China, who learned the ADTRE (analyzing, designing, teaching, revising, and evaluating/improving) instructional model, based on reflection and feedback. Using lesson planning (LP) scoring rubrics, we analyzed 113 lesson plans from 56 participants majoring in biological science and 57 in biological technology. We present the ADTRE model and discuss relationships between preservice science teachers’ academic achievement and IDC. Major findings include a positive correlation between preservice science teachers’ IDC scores and their course grades in Advanced Mathematics and Cell Biology and concept mapping skills. There was a negative correlation between preservice science teachers’ IDC and course grades in Principles of Genetic Engineering and Technology, and no significant correlations existed between IDC and course grades for teacher education courses. Our findings reveal the nature of preservice science teachers’ IDC, a potential for improvement in university teacher education curricula, and a need for further research.


INTRODUCTION
Instructional Design Competence (IDC) is an essential component of teachers' professional competence and expertise, and its importance has increasingly been acknowledged in worldwide research and policy (Tuinamuana, 2011).Official documents for teachers' professional development, especially teachers' professional standards over the last two decades, indicate an increased emphasis on the importance of teachers' IDC in England (DfE, 2011;Page, 2015), France and Germany (Page, 2015), the United States (Anagnostopoulos, Sykes, McCrory, Cannata, & Frank, 2010;NBPTS, 2001), Australia (AITSL, 2011), China (Liu & Liu, 2017;MOE, 2011), and other countries (Tuinamuana, 2011).Given the close alignment between the teaching profession and teacher education programs, there is an increased emphasis on preservice teachers' competence in instructional design and lesson planning (John, 2006).
Regarding preservice teacher preparation, research increasingly demonstrates that it is an important, yet challenging task to provide preservice teachers with opportunities to develop instructional design and planning skills before they begin their professional teaching careers (Doyle & Holm, 1998;Klein, 1991;Koehler, 2015;Ruys, Keer, & Aelterman, 2012).There are many strategies, approaches, methods, frameworks and models to achieve this objective, but developing IDC through instructional design provides an excellent opportunity for preservice science teachers because instructional design (ID) serves as a central intellectual process for developing IDC.
Many models have been used to teach ID (Branch & Gustafson, 2002).Magliaro and Shambough (2006) found that learners of ID do not always use the models given to them, but they actively and independently reconstruct models in graduate ID courses.Isman, Abanmy, Hussein and Al Saadany (2012) found that the new ADDIE (analysis, design, development, implementation, evaluation) model was strongly effective in achieving research aims, particularly for developing students' teaching skills in an undergraduate teacher education course.Nonetheless, an examination of preservice science teachers' IDC has not yet been reported.Sugar (2014) argued that ID practices do not occur in isolation.Rather, ID practices are supported by numerous elements, including preservice science teachers' beliefs (Laplante, 1996) and teaching efficacy (Cantrell, Young, & Moore, 2003), pedagogical content knowledge (PCK) (Angeli & Valanides, 2005), and knowledge of instructional planning such as cognitive demand of tasks (Bümen, 2007).Hashweh (1987) traced biology and physics teachers' subject-matter knowledge and its effect on instructional planning and teaching.Hashweh (1987) found that the teachers' prior subject-matter knowledge affected their instructional design and teaching.For instance, science teachers with detailed knowledge of their disciplines were more likely to incorporate explanatory knowledge representations, in the form of analogies and examples, into their design and enactment.
While university science courses play an important role in preservice science teachers' knowledge of content, university education courses also contribute towards the development of preservice science teachers' instructional design competence.Given that preservice science teachers take courses generally classified into foundational, major, and teacher professional education courses during their college studies, how does college academic achievement in these courses affect IDC? Weber (2015) examined how pre-service teachers in undergraduate preparatory programs learn instructional design competencies and proposed a recommended sequence to improve the practice of instructional design for online learning in teacher education programs.Yet, an examination of correlations between preservice science teachers' IDC and their academic performance has not yet been reported in the literature.Thus, the purpose of the study reported in this article is to answer the following questions: (1) What are characteristics of preservice science teachers' IDC? and (2) What is the relationship between preservice science teachers' academic achievement and their IDC?This study seeks to enrich the ID literature given that there are few IDC quantitative studies.This study's findings can also provide valuable insight for preservice science teacher education and contribute to preservice, in-service, and college teachers' IDC development.

CONCEPTUAL FRAMEWORK
The research on ADTRE (analyzing, designing, teaching, revising and evaluating/improving) based on reflection and feedback, ID models, and the analytic hierarchy system of IDC, form the foundation for our research.

Contribution of this paper to the literature
• Provides an analytic hierarchy system of IDC for evaluating preservice science teachers' IDC based on the mental model of the ID model's nature.
• Quantitatively explores the effects of an ADTRE instructional model based on reflection and feedback in teacher education.
• Reveals the real state of preservice science teachers' IDC, which has the potential to significantly improve in university teacher education; secondly, reveals several relationships between preservice science teachers' academic achievement and IDC: significant positive correlation, no significant correlation, and significant negative correlation, which suggests a weakness in university teacher education curricula and a need for further research.

ADTRE
What is the nature of ID models?In general, instructional design models, as described in the instructional system design literature, provide principles and procedures for ID and offer frameworks for guiding the design and development of successful learning activities and environments in almost all types of training and development programs.Yet, ID models can easily lead to a false characterization of ID as simply copying techniques as opposed to a rather complex intellectual process.John (2006) presented an ID model, which mimics the natural decisionmaking of an experienced practitioner in that it is not a fixed process but can fit all situations.
According to the functional definition of Rouse and Morris (1986), ID models are the mechanisms by which designers describe the purpose and form of a system, explain its function and its current state, and predict what a system might do.In fact, ID is an individual's conceptual construction and mental model (Magliaro & Shambough, 2006;Rouse & Morris, 1986).It represents how designers systematically understand a particular domain and the actions they would bring to the complex and novel instructional task and instructional situations, based on their cultural heritage, prior experiences, worldviews, and methodology.
Why construct ADTRE ID models?According to Magliaro and Shambough (2006), one of ID models' great features is continuous change depending on learning and teaching needs.John (2006) argued that published ID models, especially the dominant models of preservice teacher education require re-consideration and revision because of their linearity.The ADDIE model (analysis, design, development, implementation, and evaluation) is most frequently represented as the ID process and is generally viewed as a valuable framework for developing all types of training and development programs because of its individual and collaborative instructional development structure (Mayfield, 2011).However, the model needs revision due to its restricted approach towards learning to teach.For preservice teachers, developing ID expertise with their own style and characteristics requires learning how to use knowledge in action and spending time on teaching practice.
What knowledge is required for ID model development?Shulman (1986Shulman ( , 1987) ) stated that teaching expertise should be described and evaluated in terms of PCK, which involves relating subject matter knowledge and contexts to pedagogical knowledge.Similarly, according to other research studies of PCK (Cochran, DeRuiter, & King, 1993;Boz & Boz, 2008;Park & Oliver, 2008), ID is a creative, problem-solving process that designers integrate and understand when enacting PCK.Regarding teaching enactment, as McDonald, Kazemi and Kavanagh (2013) pointed out, the university is only one of the three settings for teacher education (the other two are P-12 classrooms and hybrid spaces).Teaching enactment practiced by preservice science teachers is an approximation of practice, or sheltered practice, where teacher educators and preservice science teachers formally and systematically ask questions and collaboratively revise plans.
ID improvement is often revised iteratively through the complementary and ongoing readjustment from reflection and feedback.As Danielewicz (2001) described, preservice science teachers' reflexivity involves their active analysis of past situations, events, and products of instructional design through critique and revision for the explicit purpose of changing thought or behavior.Feedback is essential to learning, and recent research suggests that the most effective feedback is immediate rather than delayed (Scheeler, McKinnon, & Stout, 2012).With immediate feedback, the supervisor is able to advise the teacher against performing an inadequate technique by informing the teacher of what to do, and the teacher can then perform the appropriate technique during the subsequent learning trial in the same lesson.Thus, teaching enactment, reflection, and feedback should also be included and emphasized in ID models.
What is the ADTRE model? Figure 1 offers a nonlinear, circular, and interactional model in that it emphasizes the importance of reflection and feedback as vital processes for the construction of the product (the lesson plan).The ADTRE model is conceptually defined as a visual mental model and provides an iterative decision-making process for preservice science teachers to apply in complex and diverse future teaching situations.The five phases of ADTRE are analyzing, designing, teaching, revising, and evaluating or improving.They look like leaves or petals grounded in the stem-reflection and feedback-which continuously provide critical design thoughts.In the analyzing phase, instructional tasks are determined as a result of textbook and related curriculum material analysis and learner analysis.In the designing phase, decisions are made based on the following components: content selecting, objectives making, methods selecting, media and resources selecting, and events arranging.The teaching practice phase involves teaching enactment.After the revising and evaluating phases, design is improved.
The ADTRE instructional design model integrates the advantages of the existing ID models, such as ADDIE, but it still has its own characteristics.First, the ADTRE model inherits the systemic feature of traditional ID models.Through constant and meaningful reflection and feedback, ADTRE can be not only regulated but also kept at dynamic equilibrium for a dissipative system.In terms of its use in teacher training, especially in preservice science teacher training, if we view this model as an open system rather than a closed system, like a thermodynamic system, it exists within a larger system of an external environment (an "instructional and learning environment") with which it interacts.It extracts "energy"-thinking and support from teacher educators and their peers-to construct instructional planning.It also extracts "matter"-reading and analyzing textbooks and curriculum materials and creating appropriate scripts that can be taught.
In addition, reflecting certainly plays the most important role in the ADTRE model.Every time, from a rough, fuzzy sense and awareness, with reflective behavior-and after the analysis of materials, learners, teaching objectives, content, methods, and assessments-instructional design attains the most optimal, reasonable, and effective planning.There are also a number of trade-offs, adjustments, and improvements.Reflection is not a reaction, which is a physiological, instinctive response that only leads subjects to act repeatedly and habitually.On the contrary, with reflective thinking, students continuously and critically examine their behaviors and thoughts and then construct new thoughts and behaviors beyond the original ones.Feedback is equally important in instructional design, since the result of instructional design (lesson planning or teaching) conversely affects teachers' design processes, with negative feedback that deviates from the original system goal.Instructional planning generally begins with the initial consideration of goal, content, process, and method.Therefore, feedback can achieve this optimization with authenticity and motivation without time delays.
The ADTRE model mimics the optimization of decision-making as a result of reflection and feedback.Rasmussen (1983) pointed out that three different types of decision-making exist and co-exist in a single case: skillbased, rule-based, and knowledge-based decision-making.The best ID involves all three types while designers or students utilize their IDC, ID models, and PCK for optimal design production.
How to teach with the ADTRE model in science teacher education?Reigeluth (2013), as cited in Gray et al. (2015) claimed that the traditional instructional models have been criticized because they failed to capture the complexity of the professional ID process.Lecturing on ID phases and providing the conceptual model for graduate student instruction (Magliaro & Shambaugh, 2006) is not sufficient.If we want to consider undergraduate students as ID professionals, teaching pedagogy should be modified so that learning ID involves unique ID activities and tasks aligned with particular subject areas.Accordingly, in order to improve preservice science teachers' IDC, instructional design models should be integrated into college instruction and IDC training practice.

The Analytic Hierarchy System of IDC
What is the nature of competence?Klein and Jun (2014) argued that competencies describe the critical ways in which proficiency is demonstrated.However, many researchers (Klein & Richey, 2005;Parry, 1998;Richey, Fields,  & Foxon, 2001) claimed that there are a set of related skills, knowledge, and attitudes that enable an individual to effectively perform a given occupation or job.In this study, competence, as described by Klein and Jun (2014), is applied.Competence is a critical way in which proficiency is demonstrated and a system in which multiple skills and abilities in a hierarchy can be measured through individual performance.
Instructional design competences (skills) describe a special mental ability of ID.Pedagogical design capacity primarily focuses on the performance of a special or particular IDC, such as an instructional method design skill.As decisionmaking of mental skills rather than separated procedures or ways, IDC can be considered as the choice, on some basis or criteria, between one alternative among a set of alternatives and involves several pedagogical design capacities.
Lesson planning (LP) skills must be mastered technique among the professional skills for teacher preparation programs (Martin, 1994).In teacher education professional literature and daily teaching practice, LP include curriculum or course and unit planning (John, 2006;Karges-Bone, 2000;Savage, 2015;Skowron, 2006).LP is not only a design behavior or performance, but it is also a design result or written document with special templates, models, or illustrations and graphics.
LP has been defined by Savage (2015) as the process of thinking about one's thoughts and writing down a plan for the teaching and learning of a specific group of students, in a specific place, at a specific time.Essentially, LP is viewed as mimicking the natural decision-making process (John, 2006;Squires, 1999), which requires teachers' experiences, beliefs, knowledge, and especially PCK to explore, reflect, and make decisions.Writing the lesson plan is considered a key competence for not only preservice science teachers, interns, and novice teachers but also experienced teachers (John, 2006;Karges-Bone, 2000;Savage, 2015;Skowron, 2006).Western/American and Chinese LP researchers and education practitioners share many similar understandings in terms of conceptions, types, functions, procedures, and templates (John, 2006;Karges-Bone, 2000;Liu, 2003;Savage, 2015;Skowron, 2006;Zhang, 2013).
It is worth noting that each ability in the second level of the IDC System can be divided into several skills, which constitute the sub-criteria level.The definition and indicators of each skill of IDC ability are delineated in Table 1.What needs special emphasis is that TA plays a very important role in instructional design planning, since it provides the foundation for the other design steps.The key part of the TA skill is the ability to organize and understand science content knowledge, which is also an important component of PCK, and can be presented by concept mapping (Ball & McDiarmid, 1990;Somers, 2009).The ability of RFTEI involves several skills, including reflection and feedback, teaching practice, and evaluation and improvement.The goal of fostering preservice science teachers' RFTEI ability is to develop IDC as part of teaching skills and construct a lesson plan that documents preservice science teachers' design thoughts and reveals their IDC.In the instructional process of the ADTRE instructional model, there are numerous opportunities for preservice science teachers to demonstrate ongoing reflection and feedback, teaching practices, and evaluation or improvement (RFTEI), from examination of their thoughts and actions.Since each ability is complex (Hatton & Snith, 1995;Gagné et al., 2005;McDonald et al., 2013), there is a need to develop assessment rubrics (Gagné et al., 2005), which has not been the focus of this article.How is IDC studied?IDC is a field of research on competences such as curriculum materials analysis and adaptation (Beyer, 2009;Davis, Beyer, Forbes, & Stevens, 2011), objective making (Bümen, 2007), strategies selecting (Beyer & Davis, 2009), argumentation (Knight-Bardsley & McNiell, 2015), curriculum design (Beyer & Davis, 2012), classroom discussion (Ross, 2014), inquiry (Forbes, 2009;Forbes & Davis, 2010), formative assessment (Aydeniz & Dogan, 2016), and reflection (Saribas & Ceyhan, 2015).These research areas, however, involve different aspects of IDC, and not enough attention in the past has been paid to the relations among them.
In Figure 2, LP is located at the bottom, which is the final, authentic component that represents IDC.LP can be a joyful, creative process (Karges-Bone, 2000;Savage, 2015), an art, science, and school-wide mission (Karges-Bone, 2000).Though, as an ID product, a LP document embodies thought with the quantity and quality of thinking and is viewed as analogous to natural decision-making (John, 2006;Squires,1999).Researchers (Kemp, 1971) and teacher education practitioners demonstrated that daily LP and classroom teaching could be further deconstructed into several smaller sections (subject area, unit, topic).According to Bloom's taxonomy, topics of any content area can be classified into several interconnected, smaller topics.The subtopics can be factual, conceptual, procedural, or metacognitive topics or principles (Anderson et al., 2001;Clark & Lyons, 2010).Zheng, Fu, He and Zheng (2014) proposed the CPUP model (Class Systems, Plate Systems, Unit Systems and Primitive Systems), a four-level hierarchy system model, based on the Von system science theory and observational data.
On the other hand, as a teacher training strategy, microteaching has been employed since the early 1960's (Allen & Ryan, 1969;Amobi, 2005;DeLorenzo, 1975;Remesh, 2013) and is widely accepted as one of the most important methods for providing on-campus clinical experiences for preservice teachers.Fostering preservice science teachers' IDC is easier when a whole lesson is divided into several smaller sections.Therefore, in this study we adopted 10-minute lesson planning, which is also called micro-lesson or mini-lesson because it involves planning a 10-minute lesson as opposed to a traditional 45-minute lesson.
If IDC is an interactive, hierarchical process, a system of how preservice science teachers understand ID, then it seems valuable to study preservice science teachers' characteristics and their correlations.IDC analysis not only affords instructors a concrete understanding of preservice science teachers' IDC, but it also provides them with the knowledge to create a more relevant and effective IDC.At the university level, analysis of IDC provides insight on curriculum design and reform.

Research Setting and Participants
Participants included 118 students at Shanxi Normal University in China, who were enrolled in a semesterlong, upper-level Bachelor's course called, Middle and High School Biological General Teaching Methods.Using the convenience sampling method (Gall, Borg, & Gall, 2002), these students were selected because they received the same instruction on the ADTRE model; these students studied ID 3 hours each week for 6 weeks.Out of the 118 participants, 59 students majored in biological science (Class 1 from the two biological science classes), and 59 students majored in biological technology (Class 2 from the one biological technology class).Students participated in this course during the fall semester of their junior year, from September 2015 to January 2016.For all participants, this course was their first formal study of ID, which prepares them for their teaching practicum during the spring semester of their junior year or the fall semester of their senior year, when they teach in rural schools in Shanxi Province.After ADTRE Model instruction, each preservice science teacher prepared lesson plans, and by the end of the methods course, 113 lesson plans (56 from biological science students and 57 from biological technology students) were created for analysis.

Course Description and College Instruction on the ADTRE Model
Prior to ADTRE Instructional Model instruction, preservice science teachers watched and discussed videos of biology teaching and conducted classroom observations of teaching in secondary biology classrooms (grades 7-12).They also learned about ID theories and were introduced to several ID models, such as the ADDIE model and those of Dick et al. (1996), Gagné et al. (2005), and Kemp et al. (1998).Preservice science teachers were then asked to select a topic in sequence that they were interested in from the same high school biology textbook.Using the ADTRE Instructional Model, they designed 10-minute-lesson plans, with guidance from the teacher educator and their peers in collaborative learning groups.While writing the lesson plans, they learned several abilities, which include the following: analyzing the textbook and learner characteristics, writing objectives, selecting teaching content, organizing the classroom for instruction, and choosing teaching strategies, resources, and media.
During the teaching practice phase, preservice science teachers rehearsed their plans while their preservice science teacher peers acted as students and captured videos of their lesson enactments with their cell phones.Then, with their group members, they collaboratively analyzed videos of their lessons and improved their plans.Finally, they returned to the whole class and practiced their teaching based on their improved lesson plans, while the teacher educator and their classmates role-played students.After teaching the revised lesson, each preservice science teacher received feedback and individual guidance from their professor and peers, which they used when they reflected on their lesson plans and IDC.In general, for each major, the teaching practice phase took 590 minutes, since there were 59 preservice teachers who practiced their 10-minute-lessons individually.In other words, for each 1 hour class, 4 preservice teachers practiced their teaching for a total of about 40 minutes, and there were about 20 minutes for the teacher educator's comments (for details, see Table 2).At the end of semester, all preservice science teachers took part in a university teaching skill test that high school expert teachers evaluated.Reflection and feedback, improving instruction, and IDC were central components for both teacher educators and preservice science teachers throughout the course.The details of our instructional intent and students' learning steps are also displayed in Table 2.

Scoring Rubric Development
We scored preservice science teachers' IDC based on their lesson plans, according to a rubric.To develop the rubric based on the IDC definition and skill indicators, we used the research literature (Dick et al., 1996;Gagné et al, 2005;He et al., 2016;Jefferies, 1966;Karges-Bone, 2000;Kemp, 1971;Kemp et al., 1998;Mäntylä & Nousiainen, 2014;Savage, 2015) and focus group discussions.We developed a LP analytic rubric by detailing the IDC skill indicators.The LP rubric preparation process and scoring criteria were informed by Bümen (2007) and Klein's (1991) work as well as He et al.' s (2016) instrument.Two science education professors and one expert high school biology teacher worked together to create the criteria for the rubric.
We also followed McClure, Sonak and Suen's (1999) concept mapping skill assessment to grade the skill of "understanding the instructional content systematically, logically, and hierarchically," which is part of TA.This skill includes organizing and understanding science content knowledge.We used concept mapping because research has shown that concept maps can indicate the organization and understanding of science content knowledge in a graphic, visual manner (Novak & Gowin, 1984).Concept mapping can also be used as an assessment tool (Mok, Lung, Cheng, Cheung, & Ng, 2006).Furthermore, Somers (2009) reported that concept mapping can be a strong tool for preservice teachers to organize and understand subject matter knowledge and strengthen understanding of pedagogy through reflection.Thus, we utilized a concept mapping skill assessment to evaluate preservice science teachers' IDC.Due to space limitations, we direct you to the paper written by McClure et al. (1999), which details the reliability and validity of concept mapping as a measurement instrument.In addition, we examined the initiation-response-feedback (IRF) pattern (Molinari, Mameli, & Gnisci, 2013) to grade the skill of promoting teaching with questions designed for activating students' critical thinking.Then, we tested the rubric draft on 10 preservice science teachers' lesson plans and obtained feedback from 2 science teacher education professors and 2 expert biology teachers.With this pilot test, we re-examined the performance levels and definitions of each criterion until the rubric reached its final and acceptable state.
The scoring rubric for teachers' IDC is located in the Appendix.For each category of teachers' ability, a group of subcategories, which referred to teachers' skill indicators, were defined.For example, the ability of learner analyzing (LA) contained four indicator skills, which included describing students' thinking traits or learning interests, describing students' prior conceptions or prior learning knowledge, knowing or evaluating students' learning difficulties, and knowing how to investigate students' pre-conceptions.The content of teachers' lesson plans, according to the four skills, were assessed by the researchers and a score (none -0, exact -1, and more exact -2) was assigned to each skill.The overall score of LA was computed by adding the scores of the four skills.However, the other dimensions of IDC did not share exactly the same scheme of scoring due to the complexity of skills.For example, the score of the first skill of events arranging (EA), which referred to having creative, new, and unusual thinking, was defined as creative thinking (2 marks), just review (1 mark), and none (0 marks).The total scores of teachers' abilities were then used in the further analyses.

Data Analysis
Pre-service science teachers' instructional design competence was evaluated through scoring teachers' 10minute lesson plans.We believe the 10-minute lesson includes full components of lesson teaching.As the saying goes, small as the sparrow is, it possesses all its internal organs, the instructional design competence required for a 10-minute lesson is not less than that required for a 40-45-minute lesson, which generally includes three or four 10minute lessons.On the contrary, effective instructional design competence is required to design and implement a shorter lesson plan as opposed to a longer and more traditional lesson plan.Often, it is more difficult to design micro-lessons or mini-lessons, such as those used in the flipped classroom (Bergmann & Sams, 2012) and Khan Academy experiences (Khan, 2012).For the past ten years, our university, as well as other normal universities in China, has adopted 10-minute-lesson planning as an effective practice.Some people might consider lesson content as a factor that might impact instructional design competence results.However, it is difficult for preservice science teachers to design lesson plans while they are still learning and practicing instructional design.The difficulty still lies in how to best present content to students, which requires preservice science teachers to have expert instructional design competence, which they do not yet have (Hammerness et al., 2005).However, as Hevern (2009) pointed out, Bruner (1960, 1977) argued that any subject could be taught to any child at any stage of development, as long as it is presented in the proper manner.Thus, the difficulty of the teaching content is negligible in the face of teachers' IDC.Furthermore, our analysis framework displays IDC as systematic rather than isolated, which refers to a design capability for any teaching task rather than a single task.
Reliability and validity is fundamental for any research.Credibility and content validity were achieved by using the Analytic Hierarchy System of IDC as the conceptual framework to guide the study.The content validity for the scoring rubric is based on significant western/American and Chinese theories and instructional design practices, as Scoring Rubric Development noted.In addition, in this study, the content validity for the scoring rubric has been achieved through focus group discussion between the two raters who have extensive experience in lesson plan design.Reliability was enhanced by providing the definition and skill indicators of each instructional design ability as well as an acceptable level of inter-rater reliability on the total IDC and six instructional design abilities.
We employed two raters in this study.One was a science education professor who has more than 20 years of teaching experience and teacher education experience, and the other one was a biology master's degree student with 3 years of biology teaching experience.Both individuals had extensive expertise and knowledge on lesson planning and biology education.The two raters are qualified in designing lesson plans since both have participated in professional instructional design training before they became teachers.The student has been directed by the science professor for one year on research in biology teaching.Additionally, the student took part in this research project from its inception and has participated in every step of the study since then.In order to guarantee the reliability of the results, two raters (the researchers) fully and deeply discussed every biology teaching content designed by the preservice science teachers and reached a consensus on the assessment.The result of inter-rater reliability was calculated with Spearman's rank correlation coefficient, which showed an acceptable level.Table 3 presents the coefficients.Two raters scored 113 students' lesson plans; the science education professor scored TAA, LAA, OMA, and part of EAA, while the other rater scored CSA, SRMSA, and part of EAA.The scoring served as the source of IDC data for descriptive statistics analysis.In addition, in order to study the relationship between IDC and students' academic performance, we collected and examined five semesters of these preservice science teachers' academic performance, which included general courses, required courses, and teacher education courses.In total, there were 17 courses including College English, Advanced Mathematics, Inorganic Chemistry, Organic Chemistry, Genetics, Biochemistry, Cell Biology, Microbiology, Ecology, Education, Educational Psychology, and Middle and High School's Biological General Teaching Methods.
Preservice science teachers' academic performance can be revealed in various ways, from an information era ePortfolio (JISC, 2014) to a more traditional professional knowledge test (Paulick, Grosschedl, Harms, & Moller, 2016).Subject examination scores are therefore only one method used in determining academic performance.We chose subject examination scores to study the relationship between students' academic performance and IDC because subject examinations assess preservice science teachers' PCK, which is a key prerequisite for ID mental activity and IDC development.It is unfortunate that, at present, many universities in developing countries use only subject examination scores and lack diverse academic achievement assessment methods.Shanxi Normal University is no exception.Lastly, we wanted to explore the extent to which these courses contributed to students' IDC, in that findings could provide empirical evidence for teacher education curriculum reform.

IDC Comparison between the Majors
In order to compare any significant difference in preservice science teachers' IDC between different majors, independent samples t-tests were done.The results showed that the differences between the two majors' overall IDC mean scores were not statistically significant at the 0.05 level (p =0.08).However, Class 2 (majors in biological technology) and Class 1 (majors in biological science) had statistically significant different mean scores on OMA and CSA (p <0.05).There were no statistically significant different mean scores on TAA, LAA, SRMSRA, and EAA (p >0.05).Table 5 presents the results.

Relationship between the Preservice Science Teachers' Academic Achievement and IDC
In order to determine if there was any relationship between the preservice science teachers' academic achievement and IDC, correlations analysis was done.The results indicated that there was a statistically significant correlation between students' overall IDC scores and their grades in Advanced Mathematics (enrolled during the freshman year, fall semester) (p<.05) (r=.191, p <0.05).There was no statistically significant correlation between students' overall IDC scores and grades in any of other courses.Table 6 presents the findings.
The correlation between student overall IDC scores and student grades in the required and major courses shows that there was a statistically significant correlation between IDC scores and grades in Cell Biology (enrolled in sophomore year, spring semester) (r=.244, p <0.01).There was a statistically significant negative correlation between IDC scores and grades in Principles of Genetic Engineering and Technology (enrolled in junior year, fall semester) (r = -0.216,p <0.05).There was no statistically significant correlation between IDC scores and grades in other courses (i.e., Botany, Zoology, Genetics, Biochemistry, Microbiology, Ecology, and Molecular Biology).Table 7 presents the results.
As for the correlation between IDC scores and teacher education courses, there were no statistically significant correlations between IDC and the teacher education courses (including Education, Educational Psychology, Biology Teaching Methods, and Teaching Skills Training courses), although there were significant correlations between IDC and concept mapping skills (see Tables 8 and 9).

DISCUSSION
Research substantiates the primary role that teachers play in student learning and academic success (Akiba, LeTendre, & Scribner, 2007;Darling-Hammond, Holtzman, Gatlin, & Heilig, 2005;Tatar, Tuysuz, Tosun, & İlhan, 2016).According to Stronge (2013), in order to "improve the quality of our schools and positively affect the lives of our students, we must change the quality of our teaching" (p.3).Globalization of the teacher professional standards and competencies has led to an increased emphasis on fostering teachers' qualities (UNESCO, 2014).However, as Aydin et al. (2013) pointed out, "preservice teachers have the potential to pursue lifelong professional growth as they progress throughout their career[s]" (p.904).Thus, the quality of preservice teachers requires attention and should be a key focus for university teacher education programs.Caena (2014) pointed out that, increasingly, research and policy support a holistic, dynamic, and processoriented view of teacher competences.Underpinned by this competency perspective, this study provided an analytic hierarchy system of IDC for evaluating teachers' IDC based on the mental model and nature of ID.This study also quantitatively explored the effects of an ADTRE instructional model based on reflection and feedback for 118 preservice science teachers, majoring in biological science or biological technology, at Shanxi Normal University in China.We collected 113 lesson plans from these students and analyzed them according to scoring rubrics.Specifically, we explored particular characteristics of preservice science teachers' IDC, whether or not there were differences in IDC between the two majors, and the relationship between preservice science teachers' academic achievement in different courses and their IDC.This study is aligned with the current reform efforts regarding the significance of IDC and confirms the studies that discuss how preservice teachers can be successful in acquiring and applying instructional design skills (Klein, 1991;Neale et al., as cited in Klein, 1991).

Characteristics of Preservice Science Teachers' IDC
About 50% of participants in this study attained the IDC mean.IDC components with the greatest numbers of students who attained the mean, from high to low were: SRMSA, OMA, LAA, TAA, EAA, and CSA.A little more than 60% of students demonstrated competence in selecting teaching strategies and resources and media, and a little more than 50% of the students were more easily able to write objectives.Students had slightly more difficulty with determining learner characteristics and analyzing textbooks and curriculum materials.Fewer students, about 46% and 45% respectively, had difficulty with arranging instructional events and selecting content for lessons.The above findings suggest that there is significant room for preservice science teachers to improve in their IDC.
We posit that the IDC components where students had more difficulty require more teaching experience and support.Fully understanding student needs, arranging instructional events, and selecting instructional content based on analysis of curriculum materials, are skills that are more difficult for novice teachers.On the other hand, writing objectives and determining materials needed for lessons can more easily be taught and require less teaching experience to develop.Abd-El-Khalick's (2006) study, which investigated two preservice and two experienced secondary biology teachers' global and specific subject matter structures and the relationship between these structures and their teaching practices, reveal differences between novice and expert teaching.Experienced teachers paid more attention to their students' needs and emphasized fewer details and more integrative content.Preservice teachers relied more heavily on the textbooks when teaching, having more difficulty selecting overarching themes that connected biology content.Therefore, it is possible that some of the IDC components require more expertise for competence than others, which might explain the differences in the individual IDC component results.Regarding the differences in IDC based on major, both Class 1 (biological science majors) and Class 2 (biological technology majors) had statistically significantly mean scores on OMA and CSA (p <0.05).There were no statistically significant mean scores between the two majors in the other IDC components and overall IDC scores.After all, students with different majors are in different classes, and it is possible that differences in the course curricula, or other factors, like the class culture or learning environment, could have contributed to the abovementioned differences in these preservice science teachers' IDC.However, reasons for this finding need to be explored more deeply.

Relationships between Preservice Science Teachers' Academic Achievement and Their IDC
Considering the relationship between the preservice science teachers' academic achievement and IDC, this study revealed a positively significant correlation between Advanced Mathematics and Cell Biology courses and IDC.Learning in these two courses emphasized the competences of comprehension and reasoning, and transformation and reflection, which were foundational components of teaching reform (Shulman, 1987).Shulman (1987) pointed out: "Teaching begins with an act of reason, continues with a process of reasoning, culminates in performances of imparting, eliciting, involving, or enticing, and is then thought about some more until the process can begin again" (p.13).Thus, we believe that this finding is particularly valuable because it provides strong evidence for Shulman's (1987) idea about teaching that emphasizes comprehension, reasoning, transformation, and reflection.As Shulman (1987) stated, "This emphasis is justified" even though "research and policy have so blatantly ignored those aspects of teaching in the past" (p.13).Unfortunately, even with teacher education backgrounds, we did not consider this circumstance of teaching.If we took instructional topics into consideration, we would find that nearly forty percent of the teaching topics belonged to the field of cell biology, and perhaps the preservice science teachers reviewed cell biology knowledge during instructional design.Whatever the reason, we should carefully and judiciously draw a conclusion from this finding that the reasonable thinking competence and subject matter knowledge learned in Advanced Mathematics and Cell Biology benefitted preservice science teachers' development of IDC.
There was also a significant correlation between IDC and concept mapping skills.Martin (1994) found that preservice teachers' usage of concept mapping led to lesson plans "which exhibit continuity, which are wellintegrated, and which are logically sequenced" (p.27).He argued that concept mapping is helpful for preservice teachers when developing lesson plans, especially because preservice teachers can quickly learn concept mapping skills.Conversely, there was a significant negative correlation between IDC and Principles of Genetic Engineering and Technology.This finding could be a result of limited learning time, where students were both learning material for this course and developing IDC in the same semester.
The findings were not significant for the correlations between other major and general courses and IDC.If abiding by the definition of teacher competences (Deakin Crick, 2008), IDC is viewed as a complex combination of knowledge, skills, understanding, values, attitudes and desire, leading to effective, situated actions in an instructional design mental activity.Major and general courses should be the primary origin of preservice teachers' IDC.Possible reasons for our result are that pre-service science teachers are left with fragmented knowledge because traditional teacher education is often inefficient in creating the required coherence in learned subject content.Thus, creative university teaching methods are required in order to facilitate consolidation of preservice science teachers' knowledge, including subject matter knowledge (Mäntylä & Nousiainen, 2014).
Additionally, there was no significant correlation between teacher education courses, such as Education, Educational Psychology, Teaching Skills Training, and Biology Teaching Methods.This outcome seems to support the results of previous studies that had revealed the unfortunate truth of teacher education.Several studies reported that teacher education programs are not adequately informed by knowledge or research on teachers' professional learning (RAND Reading Study Group, as cited in Aydin et al. 2013;U.S. Department of Education, 2008).Furthermore, this result also suggests the weakness in our university teachers' education curriculum and teaching methods, which needs to be explored more deeply.
These results suggest that other variables, perhaps relating to the preservice teachers themselves, could affect IDC.Hardre and Kollmann (2013) examined differences in individual characteristics that they believed could influence preservice teachers' IDC.They found that the differences that seemed to impact preservice teachers' IDC included matters of choice, some that developed over time, and others that were based on attitude or personality (Hardre & Kollmann, 2013).For instance, some preservice teachers chose content that they were more familiar with, which granted them more time, energy, and attention towards learning ID content and principles; they would not need to divide their time between mastering content and ID.Hardre & Kollmann (2013) also suggested that preservice teachers who were less likely to expand beyond their comfort zones developed IDC more slowly.Preservice teachers who were more metacognitive, reflective, and self-regulative tended to be more successful in ID.Thus, differences in individual characteristics could contribute to these disparities among particular IDC components.

RECOMMENDATIONS
Due to its complexity, there seems to be increasing convergence towards a definition, structure, and application of IDC.This study provides an analytic framework for evaluating preservice science teachers' IDC based on the mental model of instructional design and offers one of many possibilities.The analytic hierarchy system of IDC needs to be studied continually and improved to validate and explain the true nature of IDC.Furthermore, another limitation of this study is that preservice science teachers' RFTEI ability, because of its complexity, was not determined.In addition, this study assessed IDC based on the lesson plans of biological science and biological technology majors' lesson plans.Future studies could explore other methods for assessing IDC.They could also focus on other areas of science, such as chemistry, physics, and earth science, as well as different levels of participants, such as in-service, college and university teachers.Additional research can extend over a longer period in order to examine how IDC develops over time, and study novice and expert teachers' IDC.Similar to the Hardre & Kollmann (2013) study, other factors that contribute to IDC should also be examined.

Figure 1 .
Figure 1.The ADTRE Model Based on Reflecting and Feedback

Figure 2 .
Figure 2. The System of Instructional Design Competences

Table 1 .
The Definition and Indicators of IDC Skills a. Identifying the key concepts (knowledge, method, principle) b.Identifying student learning difficulties SRMS Strategies, Resources, and Media Selection consists of three decisions:(a) What kind of strategies (methods or approaches) should be employed while teaching?(b) What resources (or materials) are needed in order to accomplish the goals?), and (c) How can the key knowledge and difficulties be addressed by integrating multimedia technology into teaching?a. Chosen strategies fit the instructional content b.Resources or media are suitable for the instructional content c.Selected methods and media are helpful for a. Homework is connected to current content and supports future lesson content well b.Homework reinforces and strengthens students' learning c.Homework helps students apply and transfer new knowledge

Table 3 .
Spearman's Rank Correlation Coefficient of the Two Raters for the Rubric Components ** p<.01

Table 5 .
Descriptive Statistics for Different Preservice Science Teacher Majors' IDC Tests and Independent Samples t-test for

Table 6 .
Relationship Between General and Foundational Courses and IDC

Table 7 .
Relationship Between Required and Major Courses and IDC (N=113)

Table 8 .
Relationship Between Teacher Education Courses and IDC (N=113) IDC