Further Investigation into the Quality of Teachers ’ Noticing Expertise : A Proposed Framework for Evaluating Teachers ’ Models of Students ’ Mathematical Thinking

Because teachers cannot directly access the processes by which students construct their mathematical knowledge, Teacher Noticing, an activity that involves observing students’ work, interpreting students’ mathematical thinking about a task based on their remarks or actions, and responding to their thinking, is important to grasp students’ mathematical understanding. A possible way for teachers to develop noticing expertise is to engage in a situation focused on student thinking such as clinical interviews. However, noticing students’ thinking productively through clinical interviews remains a challenge, especially for pre-service teachers, not only because it requires a broad range of knowledge but also because of the absence of a framework to inform and evaluate the process. This paper addresses the development of such a framework for evaluating the quality of pre-service teachers’ noticing expertise in a context where students’ thinking is emphasized by removing normal classroom interruptions. It then demonstrates how the framework can be used for this purpose through three empirical examples of pre-service teachers who engaged in an intervention that involved conducting clinical interviews and analyzing students’ mathematical thinking by watching video-recordings of their clinical interviews.


INTRODUCTION
Research on teacher knowledge in mathematics education has developed extensively over the past two decades, much of it suggesting the importance of preparing pre-service mathematics teachers who have a deep understanding of content, pedagogical content and curricular knowledge (Ball, 1991;Carpenter, Fennema, Peterson, & Carey, 1988;Hill, Rowan, & Ball, 2005;Shulman, 1986), as each plays a significant role in developing effective strategies for teaching mathematics.In the domain of pedagogical content knowledge, understanding learners' thinking is considered important because instruction must connect with their present thinking in order to lead them further (Carpenter et. al, 1988;Lee, 2013;Philipp et. al, 2007).
Lee / A Framework for Focused Noticing Expertise 2 / 15 However, noticing students' thinking productively through clinical interviews remains a challenge for teachers, not only because it requires a broad range of knowledge (Cobb & Steffe, 1983;Norton & McCloskey, 2008;Steffe & Thompson, 2000), perceptions (Lee & Cross Francis, 2017), and resources (Schoenfeld, 2011;Star & Strickland, 2008), but also because of the absence of a framework that provides specified criteria for evaluating noticing skill in a situation focused on student thinking such as clinical interviews.The purpose of this paper is to develop a framework for evaluating the quality of teacher noticing expertise in a context in which students' thinking is emphasized by removing normal classroom interruptions and then to demonstrate how the framework can be used for this purpose through empirical data collected in a field experience where interventions of conducting clinical interviews and analyzing students' thinking after interviews are incorporated.

LITERATURE REVIEW
Teacher Knowledge Shulman (1986) put forward the idea that teachers' specialized knowledge of teaching differentiates them from subject matter specialists.A subject matter specialist in mathematics pursues knowledge for the development of the field, but a mathematics teacher places priority on improvement of students' understanding of mathematics.Shulman distinguishes among three kinds of knowledge that teachers need: content knowledge, referring to subject matter knowledge of mathematics; pedagogical content knowledge, which is topic-specific knowledge needed to teach mathematics; and curricular knowledge, which is used to plan grade-level educational programs for specific subjects and topics, including relevant instructional materials and affordances and constraints in their use.
Pedagogical content knowledge, which is concerned with teaching subject matter, includes four components: knowledge of curricula for a particular subject, knowledge of assessments for a particular subject, knowledge of instructional strategies for a particular subject; and knowledge of students' understandings of a particular subject.In practice, teachers use pedagogical content knowledge to comprehend networks of interrelated concepts, to represent a subject so as to make it comprehensible to students, to choose appropriate tasks for them, to understand what makes the learning process easy or difficult, and to understand students' reasoning related to specific content knowledge (Carpenter et al., 1988;Grossman, 1990;Shulman, 1986).Expertise in noticing students' thinking falls within the domain of pedagogical content knowledge.

Teacher Noticing
Noticing here is regarded as the act of observing or recognizing something, and mathematics teacher noticing is a particular type of noticing that is focused mainly on observing and interpreting students' mathematical thinking.Prior research on teacher noticing (Jacobs et al., 2010;Miller, 2011;Sherin & van Es, 2009) defines noticing in different ways depending on whether it focuses on (1) what teachers attend to, (2) their interpretation of what they attend to, or (3) their responses to students' reasoning based on what they have attended to and interpreted.Sherin and van Es (2009) focused on two main areas: what teachers notice and how they notice.The dimension of what includes both to whom and to which topics or issues teachers attend.The dimension of how captures their analytic stances (e.g., descriptive, interpretive, and evaluative) and the depth of their analysis (e.g., whether they rely on a few details, ground their interpretations in evidence, connect their analysis with learning/teaching principles, or propose alternative pedagogical suggestions).Based on these two dimensions, van Es (2011) developed a framework to assess teachers' learning to noticing student thinking in the form of a trajectory of development from Level 1 (Baseline), through Level 2 (Mixed) and Level 3 (Focused) to Level 4 (Extended).Teachers at Level 1 tend to focus on superficial classroom events (e.g., students' behavior, classroom environment, etc.), create general impressions, provide descriptive comments, and evaluate comments with little or no evidence.Teachers at Level 4, in contrast, are inclined to attend to events germane to learning such as students' mathematical thinking, infer the meaning of those events, make connections, and propose alternative pedagogical responses.Jacobs et al. (2010) and van Es (2011) both investigated teacher noticing by observing how they responded to video clips of students' behaviors, the former focusing on individual students' mathematical problem solving and

Contribution of this paper to the literature
• This paper contributes to extending existing frameworks for mathematics teacher noticing by (1) proposing a framework for evaluating the quality of teacher noticing expertise in a context in which students' thinking is emphasized by removing normal classroom interruptions and (2) demonstrating how the framework can be used for this purpose through three empirical examples of pre-service teachers who engaged in an intervention that involved conducting clinical interviews and analyzing students' mathematical thinking by watching video-recordings of the clinical interviews.

EURASIA J Math Sci and Tech Ed
3 / 15 the latter on whole-classroom instruction.Jacobs and colleagues (2010) conceptualized noticing as a set of interrelated skills termed professional noticing, which included attending to noteworthy aspects of students' mathematical thinking, interpreting their mathematical thinking based on observations, and making instructional decisions based on these interpretations.That is, Jacobs and colleagues viewed noticing expertise as a process of interrelated teachers' actions.Mason (2002) also acknowledged noticing as a key to professional development and valued it as the first step toward action.He described two forms of recording observations that could help teachers become more sensitive what they see: accounts-of and accounts-for.Accounts-of indicates recording an event as it would be seen by other observers without including emotion or judgment, which is similar to Jacobs et al. (2010)'s notion of attending to, while account-for refers to providing "interpretation, explanation, value-judgment, justification, or criticism" (p.40), which is comparable to Jacobs et al. (2010)'s concept of interpreting.Similar to the third component of Jacobs et al. (2010)'s framework of professional noticing, making instructional decisions, Leatham, Peterson, Stockero, and van Zoest (2015) conceptualized a framework they titled Mathematically significant pedagogical Opportunities to build on Student Thinking (MOST), which can be described as the intersection of three characteristics of important components in teaching mathematics: mathematically significant content, pedagogical opportunity, and students' mathematical thinking.They recommended first attending to students' mathematical thinking, then focusing on whether the students' thinking is mathematically significant so that it can advance students' development of critical mathematical ideas, and finally considering, at the moment of teaching, pedagogical ways to build on students' significant mathematical ideas to support their further progress.When compared to other frameworks related to noticing, Leatham et al.'s framework is meaningful in that it explicitly addresses the significance of students' mathematical ideas as an important aspect of noticing expertise.This clarification can provide a lens to help teachers develop mathematically productive use of students' thinking.

Factors Impacted Teacher Noticing
Several researchers have found that effectively noticing students' mathematical thinking requires both the ability to focus on important events and knowledge of what is mathematically significant in students' messy strategies (Jacobs, Lamb, Philipp, & Schapelle, 2011;Leatham et al., 2015).Schoenfeld (2010) also emphasized the importance of teachers' knowledge, resources, and orientations in noticing students' thinking and making decisions to support the students' learning.In a similar vein, the important role of teachers' perceptions of the use of students' mathematical thinking was highlighted in the finding that the deeper the awareness of the use of students' thinking teachers demonstrated in open-ended survey, the more effective the noticing skills they showed during a taskedbased interview (Lee & Cross Francis, 2017).On the other hand, some researchers (Dreher & Kuntze, 2015;Jacobs et al., 2010) found that in-service teachers tended to demonstrate more advanced noticing ability than pre-service teachers, suggesting that teaching experience influences the development of noticing skills, although, as Dreher and Kuntze (2015) pointed out, teaching experience is not always aligned with teachers' practices.They noted that the relationship between teaching experience and noticing skill is complicated, and that noticing expertise seemed to be closely related to teachers' ability to draw on different components of professional knowledge.Another study (Lee & Choi, 2017) supported this possibility by suggesting that introducing focal points to pre-service teachers would contribute to the development of productive noticing skills despite their lack of teaching experience.All the studies described here confirm that teachers need deep understanding of mathematical knowledge as well as skills to decide instructional moves that support students' learning.

Ways to Improve Teacher Noticing
Although researchers have conceptualized noticing in various ways, most have considered teacher noticing as a key component of teaching expertise because it influences the quality of mathematics instruction (Jacobs, Lamb, & Philipp, 2010;Schack, Wilhelm, & Fisher, 2017;Sherin, Jacobs, & Philipp, 2011).For this reason, those studies have investigated various strategies for improving teachers' noticing skills through professional development.For example, teachers have been asked to describe what they notice in another teacher's instruction video (Colestock & Sherin, 2009;Kersting, 2008) or to retrospectively recall what they noticed while watching their own teaching videos (Ainley & Luntley, 2007).Other researchers have tried to improve teachers' noticing expertise by asking them to watch and discuss excerpts of their own teaching or others' videos with their colleagues as a group (Lee & Choi, 2017;Sherin & van Es, 2009).Moreover, several teacher educators have implemented interventions in their methods or content courses to improve pre-service teachers (PSTs)' noticing expertise, also often using videos as a tool for PSTs to analyze students' mathematical thinking.Star and Strickland (2007) found that such intervention helped improve PSTs' ability to notice the salient features of secondary students' thinking.In their study of 131 prospective teachers and practicing teachers, Jacobs et al. (2010) found that prospective teachers struggled with all three components of Lee / A Framework for Focused Noticing Expertise 4 / 15 noticing, that teaching experience enhanced practicing teachers' abilities to attend to students' strategies and understandings, and that professional development provided support for all three skills.Schack, Fisher, Thomas, Eisenhardt, Tassell, and Yoder (2013) developed a five-session module utilizing video excerpts of diagnostic interviews of students' mathematical problem solving, which progressively interrelated the three components of professional noticing, and implemented the module with 94 elementary PSTs in mathematics methods courses at three institutions.The pre-and post-assessments showed that the module was effective in developing PSTs' professional noticing skills.Roller (2016) also found that viewing videos of their own co-teaching an audience of peers in a microteaching setting promoted secondary PSTs' noticing skills.
In addition, Fernandez, Linares, and Valls (2012) examined how and when PSTs participate in on-line discussions implemented to improve their noticing skills and found that participating was effective in improving their ability to collaboratively interpret students' mathematical thinking.Fernandes (2012) investigated the impact of an intervention in which mathematics PSTs conducted task-based interviews with English Language Learners (ELLs).The author asked pairs of PSTs to conduct video-recorded task-based interviews using four measurement tasks with two ELL students and found that the intervention promoted PSTs' awareness of ELLs' needs and challenges and led them to employ strategies that were aligned with best practice for teaching ELLs.
In the studies discussed above, the emphasis has been on helping PSTs focus on mathematically important events such as students' thinking.Noticing is a critical skill for understanding students' thinking, but in a classroom situation where multiple events are occurring simultaneously, noticing students' thinking may be hard for teachers, especially for novices and PSTs because of other distractions such as behavior and classroom management issues.Thus, providing PSTs with an environment in which they can focus on students' thinking, such as teaching experiments, may be a key early component of teacher preparation.According to Amador (2016), teaching experiments "provide a context for learning to notice because of the emphasis on students' thinking and the reduction of classroom interruptions" (p.220).

Teaching Experiments and Model Building
Amador (2016) describes teaching experiments "as extending clinical interviews to encompass the scientific process of building explanatory and predictive models of students' mathematical understanding" (p.220).Steffe and colleagues (Cobb & Steffe, 1983;Steffe & Thompson, 2000;Steffe & Wiegel, 1996) defined models in mathematics education as "constellations of theoretical constructs that represent our [i.e., the teachers' or researchers'] understanding of children's mathematical realities" (Cobb & Steffe, 1983, p. 83).This conception distinguishes teachers' practice-based models from second-order models, which are drawn from theoretical constructions by researchers who focus on developing explanatory frameworks.Rather, teachers build models to describe particular students' thinking and use the information to make instructional decisions.Thus, in this paper a model is defined as "a set of generalizable statements that represent one's [teachers'] understanding of how a student is reasoning about a particular mathematical concept, which might include statements of predictive value" (Lee, 2013, P. 19).That is, as a way of improving noticing expertise, teachers are expected to build experiential working models as critical precursors to second-order models rather than construct second-order models themselves.
Building models is important for choosing instructional tasks, asking appropriate questions, orchestrating classroom discussions, adapting instruction to students' needs, and diagnosing and remediating students' learning difficulties.In this respect, building models can be regarded as one of the most important factors in developing pedagogical content knowledge (Confrey, 1990;Norton & McCloskey, 2008;Steffe & Wiegel, 1996).From a constructivist perspective, building models is an essential process for understanding students' thinking (Steffe & Thompson, 2000;Steffe & Wiegel, 1996).According to constructivists, learning occurs when students modify and reorganize their current thinking to construct new ways of knowing (von Glaserfeld, 1995).Thus, being aware of students' current ways of thinking is important for effective mathematical teaching.However, because teachers cannot have direct access to learners' mathematical knowledge, they need to build models of students' thinking to understand their reasoning.Also building models to interpret students' performances can provide a valuable resource for effective lesson planning (Confrey, 1990).Additionally, the models formulated by interactions with students provide a framework for teachers or researchers to engage in further interactive mathematical communication with other students as well as with other teachers or researchers who are concerned about students' mathematical thinking (Cobb & Steffe, 1983).
The models used in mathematics education are created by analyzing students' mathematical thinking, both retrospectively and prospectively.A teacher or researcher begins modeling a student's understanding by asking what the student thinks, so that the student's actions in a given situation make sense from his or her own viewpoint (Hackenberg, 2005).To build a model, the teacher or researcher puts him or herself in the student's position and examines the student's mathematical thinking from that perspective through careful observation and one-on-one interaction (Confrey, 1990).As observations of and interactions with each student are accumulated, the teacher is able to see patterns across students' actions and verbalizations.The teacher then becomes reflectively cognizant of EURASIA J Math Sci and Tech Ed 5 / 15 patterns in the observation by identifying persistent constraints1 on students' ways of thinking.Through this process, the teacher eventually arrives at an explanatory framework, which is then used for new purposes.This modeling process was adapted to the particularities of the participant group (i.e., PSTs) and the field experience setting (i.e., science and mathematics field experience).The process by which PSTs in this study created models of young children's thinking is illustrated in Figure 1.

Model Building and Teacher Noticing
The extent to which noticing occurs is critical in building models of students' thinking.As stated by US Math Recovery Council (2005), the value of building models is that the process involves: ongoing assessment through careful observation [c.f.attending to students' thinking from the notion of noticing suggested by Jacobs et al.], hypothesizing about a student's current knowledge and strategies [c.f.interpreting students' thinking], and selecting learning activities closely attuned to the child's current reasoning [c.f.making an instructional decision to support students' learning] (p. 6).
For this reason, some researchers have investigated building models through clinical interviews as a way to improve teacher noticing because these activities offer a context in which to examine students' thinking while learning to notice (Amador, 2016;Norton & McCloskey, 2008;Steffe & Thompson, 2000;Weiland, Hudson, & Amador, 2014).For example, Weiland et al. (2014) found that PSTs developed competent questioning skills while they engaged in clinical interviews and analysing students' thinking by building models of their scientific and mathematical thinking.Amador (2016) used elements (e.g., analytic stances, the depth of analysis, etc.), which are listed under the dimension of how from van Es's (2011) framework in order to investigate how the professional noticing expertise of four mathematics teacher educators developed through building models of their students' mathematical thinking.She found that novice teacher educators provided evaluative comments but failed to deeply analyze students' thinking and make strong connections between students' thinking and teaching/learning principles.Her research also suggested the need for a framework to evaluate the quality of teachers' noticing expertise in a situation focused on student thinking.
In this paper, I extend the notion of noticing to include model building ability.To facilitate the development of this ability, I propose a framework designed to evaluate noticing expertise demonstrated in a context in which normal classroom interruptions are reduced and the emphasis is on students such as clinical interviews or teaching The framework provided here includes some sub-components from the how dimension of van Es's (2011) framework, but these are extended by considering the features of models created based on clinical interviews, so that the proposed framework can be used to evaluate teachers' models of students' thinking, which are closely related to their noticing expertise.
According to Steffe and Thompson (2000), the quality of models is determined by the number of interactions with students on which they are based and their use of such tools as theoretical constructs to account for students' thinking.As they explain, a good model builder first sees "a pattern in students' models of explanation from interaction with students" that "must be expanded to abstracting reflectively the operations that one applies in constructing explanations" (p.294).Thus, as explained by Cobb and Steffe, (1983), a good model includes both generality in that it "should be general enough to account for other children's mathematical progress" and specificity to be "specific enough to account for a particular child's progress in a particular instructional setting" (p.91).That is, a good model shows consistency both throughout the whole experience of one student vertically (specificity) and across a group of students horizontally (generality).For this reason, it was determined that for a framework to evaluate teachers' models of students' thinking, it should include two main criteria: descriptiveness, a version of specificity, and generality.Descriptiveness refers to how well a teacher has described a student's thinking based on evidence from interactions with the student.Generality concerns whether the teacher has formulated generalizable statements from this description.

Development of Sub-criteria of Descriptiveness
The first sub-criterion of descriptiveness is thoroughness.To get a clear sense of what is happening in an interaction between a teacher and students, it is important for the teacher to consider a range of data such as interview segments, observations of students' actions, and examples of their written work, and to select key parts to support his/her model.If models are to be shared with other practitioners or researchers, it is especially important to establish the context of the interaction (Boaler & Humphries, 2005;Lampert & Ball, 1998) by describing interview goals, tasks posed to students, and other critical information that only the model builder can provide.
The second sub-criterion of descriptiveness is whether models are based on enough evidence.Norton, McCloskey, and Hudson (2011) developed a rubric to assess PSTs' knowledge of students' mathematical thinking as represented by models they derived from video-based predictions.As one of the components of the rubric, "model" measures the quality of PSTs' models according to how well they have used evidence to support their inferences of what or how the student in a video was thinking.Confrey (1990) and Steffe and Thompson (2000) also emphasized the use of evidence from observations and one-on-one interactions in order to build powerful models.Based on these studies, being evidence-based was determined as a sub-criterion of descriptiveness to evaluate the extent to which teachers justified their models by providing explicit evidence of students' work.
These two sub-criteria for descriptiveness are also reflected in van Es's (2011) framework by focusing on whether teachers refer to specific events and interactions as evidence and whether they elaborate on events and interactions, which are mentioned as one of the characteristics of teachers who demonstrate higher levels of noticing skills.However, van Es's framework does not consider the generality of teachers' interpretation of students' thinking as a criterion to assess teacher's noticing expertise, which is important for providing a lens through which they can predict and productively use students' mathematical reasoning for lesson planning.

Development of Sub-criteria of Generality
As mentioned above, a model in this study was defined as a set of generalizable statements to capture students' thinking.Here, the phrase "generalizable statements" refers to statements of ideas that teachers could generalize as a common feature of students' thinking.More specifically, generalizing 2 in this context entails at least one of three complex actions: identifying commonality across students' specific examples, extending teachers' reasoning beyond the description from which it is derived, and drawing broader statements from specific examples.Such generalizable statements can help account for other learners' mathematical thinking.Accordingly, the extent to which teachers provided generalizable statements was one sub-criterion for generality.
Faithfulness was another sub-criterion for generality.A model is a plausible explanation of students' thinking, and thus it can never be verified as definitively true or false.However, the viability of a model can be assessed by confirming whether generalizable statements are consistent with evidence (Cobb & Steffe, 1983).The criterion of faithfulness has two aspects.First, faithfulness assesses whether generalizable statements seem consistent with teachers' descriptions of students' thinking in a selected data source.Second, faithfulness assesses whether generalizable statements fit well with the evidence from all data sources.Cases of problematic faithfulness might occur in three ways.The first way is that a model has inconsistencies or misinterpretations in relation to one selected part of the data (e.g., interview video prediction, students' work, or observation).The second way is that the model exhibits inconsistencies or misinterpretations when judged across all data sources.In other words, teachers' models are assessed based on whether teachers have accurately identified and interpreted all evidence related to their specific claims about what a student knew or did not know, or whether they have missed some critical information to support their claims or misunderstood some evidence.The third way in which faithfulness is problematic is that teachers have not selected appropriate evidence that captures the key elements of students' mathematical thinking.Teachers who do a better job at building models also do a better job of picking representative evidence by considering all data sources (Steffe & Thompson, 2000).Thus, whether teachers have selected the best excerpts to accurately demonstrate students' understanding and applied them appropriately was included as an indicator to evaluate faithfulness of teachers' models.
Although not explicitly included in the framework for assessing PSTs' models proposed in this study, three components of sophisticated models -predictability, research-based knowledge, and awareness of constraints in making a generalizable statement -can also be considered in further development of the framework.Steffe and Thompson (2000) stated that the quality of models is determined by how well they account for students' thinking and predict students' thinking in a similar or more advanced situation.Also, Norton and McCloskey (2008) found that the use of research-based frameworks in professional development supports teachers' ability to build more viable models of students' thinking.In their study, practicing teachers continually refined their initial models by referring to research-based frameworks in drawing inferences from their students' current activity to predict their subsequent activity.
Furthermore, considering the observation of Cobb and Steffe (1983) that making a good model involves "dialectical interaction between the theoretical and empirical aspects" (p.91), it is important for teachers to recognize what kinds of additional information they need in order to test the viability of their models.In the model building process, unexpected observations can lead to reformulation of theoretical constructs, and conversely, a theoretical reformulation can lead to interpreting previous observations in a novel way (Cobb & Steffe, 1983).Thus, being aware of constraints on making generalizable statements by addressing the need for additional questions and tasks is worth being included in the criteria of a sophisticated model.However, because the PSTs in this study had little or no experience with either building models of children's thinking or teaching children mathematics, it was unreasonable to expect them to build sophisticated models that included predictability, research-based knowledge, and acknowledgement of constraints on making general statements.As novices, they could reasonably be expected to make general statements that have predictive value, and this predictability, which should enable others to use the model to predict how that student would respond to a similar question or task.In this regard, even though PSTs were not expected to show this level of generality, if some PSTs' models did, I also commented on it in my analysis.
Taken together, the criteria for evaluating the quality of teachers' focused noticing expertise are provided in Table 1.
Lee / A Framework for Focused Noticing Expertise 8 / 15

EXAMPLES OF ELEMENTARY PRE-SERVICE TEACHERS' FOCUSED NOTICING EXPERTISE DEMONSTRATED IN THEIR MODELS OF STUDENTS' THINKING
The three examples shared in this section were selected to show how the proposed framework can be used to evaluate PSTs' noticing expertise by analyzing their models of learners' thinking and to examine differences across their models.These examples were produced in a field experience project focused on understanding young children's mathematical thinking through clinical interviews and building models (IMB, 2010).Prior to teaching in the elementary school classroom in which they were placed, PSTs were engaged in an iterative model building intervention (cf. Figure 1) in which they were paired to interview a pair of children in the process of performing mathematical tasks.These interviews were video-recorded for the PSTs to use as resources for formulating models supported by evidence from video clips of the children's problem-solving behaviors.To create models, PSTs first chose one or two segments that best showed the reasoning of the students that they interviewed.Because the PSTs were assigned to different grades and classrooms, the content varied across the video-recorded clinical interviews depending on the topics being covered in the assigned classrooms.Therefore, although examples from the same content would have been preferable for comparing the three levels, owing to the logistical limitations of the field experience, I selected examples from three PSTs who interviewed children in lower grades (kindergarten through second grade) about a topic relevant to counting.In the following discussion, the first example is of high-level focused noticing expertise, the second of mid-level, and the last of low-level.

Example 1: High-level PSTs' Focused Noticing Expertise Shown in a Model of a Student's Thinking
High-level focused noticing expertise demonstrates all four criteria, meaning it is thorough, evidence-based, includes generalizable statements, and demonstrates faithfulness, in addition to which it incorporates at least one component from sophisticated level models.That is, PSTs provide detailed background information and explicit evidence that support their models of a student's thinking.Moreover, they present statements indicating generalizable features of the student's thinking, which are faithful to their descriptions of students' work based on focal video clips and also to evidence from the entire interview video.In particular, the generalizable statements in their models include insights applicable to describing the majority of students' thinking in that they are consistent with the findings of prior research.

Descriptiveness Thoroughness
This criterion is about whether a teacher has provided enough background information (context and tasks).(Norton, et al., 2011) Evidence-based This criterion is about whether a teacher has included explicit evidence that supports the model from details of the student's work in a segment of data selected by a teacher.(Norton, et al., 2011)

Generalizable statements
This criterion is about (1) whether a teacher has provided statements that indicate a generalizable feature of the student's thinking, rather than just a description of a student's solution to a specific problem and also (2) whether the generalizable statements are applicable to describing other students' thinking.(Cobb & Steffe, 1983;IMBP, 2010)

Faithfulness
This criterion is about (1) whether generalizable statements are faithful to a description of a student's work from a selected part of the data and (2) whether they fit with evidence from all data sources.(Steffe & Thompson, 2000) Components from sophisticated level models

Predictability
This criterion is about whether others can use the model to predict how that student would respond to a similar question or task.(Norton & McCloskey, 2008) Research-based knowledge This criterion is about whether a model explicitly depends on existing theoretical constructs, such as counting schemes, developed by other researchers.(Cobb & Steffe, 1983;Steffe & Thompson, 2000) Awareness of constraints in making a generalizable statement This criterion is about whether a teacher recognizes what kinds of additional information he/she needs in order to create a more viable model.(Cobb & Steffe, 1983) EURASIA J Math Sci and Tech Ed 9 / 15 make ten out of her Unifix cubes, Grace made two groups consisting of five cubes each.Also, when Joy asked how many groups there were and how many cubes in each group, Grace was able to state that there were two groups and five in each group.Then when asked to make another combination of ten, Grace made five groups of two using cubes, and she also knew that there were five groups and two in each group.In addition, when Joy asked how many cubes she had, she whispered "one, two, three, four, and five while pointing at each group of two and then answered there were 10 cubes there.Also, when Joy asked her to count the cubes by twos, Grace said "two, four, six, eight, ten." As evaluated using the proposed framework, Joy's model fulfilled descriptiveness and generality, as well as one criterion from sophisticated models.In terms of descriptiveness, it met the criterion of thoroughness quite well.After articulating that the goal of the interview was to assess students' knowledge of comparing, combining, recognizing, and representing numbers one through ten, Joy described the interview task, which was to show two different ways to make combinations of ten, and the context, the provision of Unifix cubes as problem solving aids.Joy's model provided sufficient background information to be understandable in its representation of Grace's knowledge of how to combine the same numbers to make ten and of how to recognize and represent ten as the sum of some number of same-number pairs.Her model also satisfied the criterion of being evidence-based in that she supported it with two relevant examples of the student's work.The first example from the video segment showed Grace's immediate response of "five and five" to the question, "how many different ways can you make ten?"The second example showed Grace pausing and then making five groups of two with her Unifix cubes when asked, "Are there any other ways to make ten?" In terms of generality, Joy provided generalizable statements in her model of Grace's thinking.For example, Joy stated, "if presented with ten Unifix cubes, Grace would be able to make two different same-number combinations to get ten.She also can count by ones to get the total number of ten.However, Grace cannot come up with different number combinations for ten such as 6+4, 2+8, 1+9 and 7+3."These statements could be considered applicable not only to Grace's but to other students' thinking because they were consistent with the results of prior research (Sarama & Clements, 2009).When asked to find a combination of the number 10, many students would first answer 5+5 because humans have 10 fingers, five on each hand, so the combination of 5+5 would be intuitive to them.Also, numerical children usually have a good sense of doubling numbers (Ginsburg, 1989;Sarama & Clements, 2009).
Joy's model was also faithful according to both aspects of faithfulness.That is, her generalizable sentences were well related to her evidence from the focal interview video clips, which showed that Grace demonstrated two different ways of combining numbers to make 10, first using the strategy of doubling a number (5+5) and then using the strategy of counting by twos (2+2+2+2+2).These generalizable sentences supported Joy's conclusion that Grace was familiar with making combinations of the same numbers when asked to make 10.
In addition, Joy's model was considered faithful to the evidence from her entire interview video.It showed that Grace was able to keep track of how many times she had counted by two or by five in an activity, which was confirmed by characteristics Grace exhibited throughout the interview video.For example, in another part of the video, after she had made her two combinations for ten and shown that she understood how she had formed them, Grace was asked how many cubes she had, and she answered there were 10 cubes by whispering one, two, three, four, and five while pointing at each group of two, indicating she could do "double counting."This observation was confirmed when she was asked to count the cubes by twos, and she responded two, four, six, eight, ten.Thus, Joy's model showing that Grace had the concept of grouping and ability to count by twos fit with other evidence from entire interview video.The goal of this interview was to learn more about students' knowledge of comparing, combining, recognizing and representing numbers one through ten.This was done by giving students Unifix cubes and asking them "How many different ways can you make ten?"In the chosen clip Grace shows two different combinations of how to make ten.
In the video segment when Grace is asked, "how many different ways can you make ten?" she automatically does the five and five.The interviewer asks "Are there any other ways to make ten?"After pausing, Grace makes five groups of two with her unifix cubes.This shows that Grace is only familiar with making the combination of ten with the same numbers.This can be achieved through doubling numbers such as five plus five or repeated addition or counting by 2's such as 2+2+2+2+2 to get to 10.
If presented with ten Unifix cubes, Grace would be able to make two different same number combinations to get ten.She also can count by ones to get the total number of ten.Her same number combinations are by grouping five and five.Then her second combination is five groups of two (2+2+2+2+2).However, Grace cannot come up with different number combinations for ten such as 6+4, 2+8, 1+9 and 7+3.
Lee / A Framework for Focused Noticing Expertise 10 / 15 Finally, Joy's model of Grace's thinking demonstrated one of the components of sophisticated models, predictability.In her model, Joy used an if-then sentence to predict how her student would respond to a similar task: "if presented with ten Unifix cubes, Grace would be able to make two different same number combinations to get ten.She also can count by ones to get the total number of ten."

Example 2: Mid-level of PSTs' Focused Noticing Expertise Shown in a Model of a Student's Thinking
Mid-level focused noticing expertise mainly demonstrates the three criteria of thoroughness, being evidencebased, and having generalizable statements.Some PSTs in this study included a component from sophisticated models.However, their noticing expertise had some limitations in faithfulness in that their models were faithful to descriptions of students' work based on focal video clips but did not fit other evidence in the entire video.Also, some PSTs based their models on video clips that did not capture key features of the students' mathematical thinking.
Table 3 represents Mary's mid-level focused noticing expertise as shown in her model of Mila, a first grader.When Mary asked Mila to add 10 cubes to the original collection, she started from the beginning to count one block at a time while pulling it with her finger and repeated this action when asked to add 10 more cubes.In addition, when asked to count 30 blocks, Mila counted the cubes one by one up to 20, but after 20, her number word did not always correspond to only one cube, and she finished by answering 29.When asked to count again, she counted correctly one by one to 20, but she touched two cubes when saying "twenty five," and also she counted two connected cubes as one cube, this time answering 28.Moreover, when asked to count three hidden cubes and five visible cubes, Mila used five fingers on one hand to represent the visible cubes and three fingers on the other hand for the hidden cubes and counted all eight fingers from the beginning.
Mary's model of the mathematical thinking of Mila fulfilled the standards of descriptiveness and generality in the proposed framework, and included a component of sophisticated models, being research-based.However, her model showed some weakness in the criterion of faithfulness.More specifically, Mary's model met the criterion of being evidence-based because it drew on specific examples from the interview video.In her model, Mary described that Mila was able to count objects that she could see and touch.She also noted that Mila needed to use her fingers as manipulatives to represent and count the three hidden blocks and add them to the five visible blocks, all of which she counted one by one to reach the total number of eight.
In terms of generality, Mary provided generalizable statements in her model.Mary stated that Mila was able to count blocks or fingers only if they were visible.More specifically, Mila was able to add visible blocks and hidden blocks if she first represented the latter with her fingers and counted them all one by one from the beginning, evidence that she could not count on.Relying on physical objects and recounting from the beginning when objects are added to an original collection are behaviors commonly seen in students who do not construct the meaning of numerosity (Olive, 2001;Steffe et al., 1983).These students tend to depend on sensory-motor operations with tangibles such as blocks and fingers in order to count and to start over when objects are added.In this regard, Mary's generalizable statements could be used to describe other students' thinking.The focus on the clip was to better understand what types of counters students are and how they are evolving as counters to better prepare our lessons to include all types of counters.From the first transcribed segment, we were able to learn that Mila is able to count to at least 30, but still must count physical objects or the movements of those objects to keep track of what she has counted in order to get to the correct answer.When bringing in more blocks, she is still only able to count by 1's and starts from the very beginning instead of counting on.The second segment shows that Mila understands how many 'units' a number represents.Even if the 3 blocks aren't shown, she is able to use manipulatives (her fingers) to represent what is hidden and then add that to what manipulatives she can see (the blocks).
Mila is able to count the blocks that are physical objects that she can see and touch.However, Mila has to create manipulatives to understand what the 3 blocks under the cup look like in order to add that to the 5 blocks on the outside.She holds up 5 on one hand for the blocks on the outside and 3 on the other hand for the blocks under the cup she can't see.Now that she is able to see the total number of fingers (which represent the total number of blocks) she is able to count how many fingers she is holding up.
The model and evidence prove that Mila is only comfortable counting when she can use manipulatives to represent numbers and that she must start with 1 and count until everything is counted.Mila is still not able to count on because when she counted her fingers she didn't recognize that 1 full hand equals 5 and then she would just count 6, 7, 8. Mila knows very well the order of numbers and how to count, but when she is counting the quantity of one group plus the quantity of another group she basically has to combine them into one large pile and just count the whole thing (counting all).Mila didn't yet recognize how to 'count on' from numbers when adding numbers together.She views adding as putting two parts together and counting the total, rather than adding on a certain number to a previously known number.

/ 15
However, Mary's model did not completely fulfill the criterion of faithfulness in that it conformed to her two selected video segments but not to some parts of the entire clinical interview.In the first segment, when asked to add 10 cubes to the original collection, Mila counted from the beginning one by one and repeated this action when asked to add 10 more cubes.In the second segment, Mila counted fingers from the beginning to add three hidden and five visible blocks to find a total number of cubes.Thus, Mary's model, which indicated that Mila needed physical objects to represent quantities and counted them by starting over from beginning when objects were added rather than count from the given numbers, seemed to capture central aspects of the student's thinking.
However, Mary's claim that Mila was able to count to at least 30 was not faithful to some evidence from the entire video.When asked to count 30 blocks, Mila touched each block while counting one by one, as the model stated, but after 20 her number word did not always correspond to an object, which led her answer 29 instead of 30.In the second trial, Mila counted correctly one by one to 20, but ended up with answering 28 due to failing to match a correct number word to each cube after 20.From this evidence, it could be said for certain only that Mila was able to count forward to 20.
Third, Mary's model of children's thinking demonstrated a component from sophisticated models.By setting the goal of understanding what type of counters students are, Mary drew on two important assessment constructs in the theory of counting or early numeracy (Steffe et al., 1983): (1) whether a student is able to count objects placed out of the perceptual field, and (2) whether a student is able to count on when objects are added to an initial collection.

Example 3: Low-level of PSTs' Focused Noticing Expertise Shown in a Model of a Student's Thinking
Low-level focused noticing expertise fulfills the criterion of being evidence-based but shows limitations in thoroughness, generalizability, and faithfulness.Noticing at this level may lack of thoroughness because the interview context is not adequately described.In addition, PSTs at this level often fail to provide generalizable statements.Although they include examples of students' activities in their models, they focus only on reporting the students' work in detail, often using quotes from the interview videos.Also, at the low-level, the criterion of faithfulness often cannot be assessed because without the provision of generalizable statements, there is no basis for determining faithfulness.
Table 4 represents John's low-level focused noticing expertise in his model of the mathematical thinking of Keywaine, a second grader.John's clinical interview was focused on understanding how students use addition strategies in story problems.In particular, John investigated whether students can use "doubles and near doubles" strategies in addition, which refers to doubling given numbers when asked to add the same numbers or two almost equal numbers.In the interview, Keywanie solved the first problem by using the standard computational algorithm.When asked to solve 10+10, Keywaine mentally computed very quickly.Then when John asked how he knew the anwers were 20, Keywaine answered "because one plus one is two and zero plus zero is nothing."When John asked whether there were other strategies to solve 10+10, Keywanie said, "double fact" after hearing interviewer's cue, "what is called adding the same number twice?"In the second problem, when asked to solve 10+11, Keywaine answered 21, again using the standard computational algorithm, "one plus one is two and zero plus one is one."When asked other strategies to solve 10+11, he initially did not answer anything but when asked what "almost a double but not quite" would be called, after being reminded of using doubles for the first problem, Keywaine answered "a near double."Then he explained how the concept of "near double" worked for the problem 10+11 by doubling 10 and then adding 1 to obtain 21 as the sum.
John's model satisfied the criterion of being evidence-based because he provided two detailed examples of the student's work with the questions posed to the student.In the first segment, John described how Keywaine solved the problem of 10+10 in his head and with John's help how he came up with the term "double strategy," which is an alternative strategy to solve 10+10.In the second segment, John described how Keywaine solved 10+11 by using a near double strategy although the idea was initiated by John's prompt.
However, John's model partially fulfilled the criterion of descriptiveness.That is, John's model showed a lack of thoroughness in that he provided insufficient background information.Although he articulated the goal of the interview as discovering how students use doubling and near-doubling as math solving strategies, he did not specify the interview context but simply indicated what was shown in the clip without giving such information as the location and context of the selected clip within the entire interview video.Most importantly, John did not provide any generalizable statements in his model of Keywaine's mathematical thinking.

Figure 1 .
Figure 1.The process by which a model is created

Table 2 represents
Joy's high-level focused noticing expertise as shown in her model of the mathematical thinking of Grace, a kindergarten student.When asked in the clinical interview how many different ways she could

Table 1 .
Proposed Framework for Evaluating the Quality of Teachers' Focused Noticing Expertise Main

Table 2 .
Joy's Model of a Student's Mathematical Thinking

Table 3 .
Mary's Model of a Student's Mathematical Thinking