Data Explorations : Secondary Students ’ Knowledge , Skills and Attitudes Toward Working with Data

The Data Explorations in Ecology Project was a professional development and research project designed to address data literacy issues in secondary science classrooms. Curricular modules focusing on locally relevant environmental issues were developed and implemented to support students in gaining proficiency with a variety of data exploration practices. The research focused on highlighting students’ knowledge, skills, and attitudes toward these practices. The findings indicate that students across grade levels are able to demonstrate proficiency with some data exploration practices but have difficulty applying these skills to the creation and evaluation of data-driven arguments about the environment. The findings also indicate that despite the difficulties secondary students have with some data exploration practices, some still enjoy and appreciate the usefulness of working with data. This study introduces a conceptual framework that illustrates some of the data exploration activities that occur at different phases of inquiry and critique processes.


INTRODUCTION
National and international scientific and education groups recognize the need for a scientifically literate citizenry that can apply science to important environmental issues in order to make sound decisions and create policies that protect ecosystems (AAAS, 1993;ACERE, 2009;NRC, 2012).Because of this, citizens are being asked to become critical consumers of information so that they can make pivotal decisions about the environment based on an evaluation of arguments in their schoolwork, in the media, and as they vote and make everyday choices.However, the arguments they encounter often have weak scientific reasoning, limited supporting evidence and many types of bias.In light of this challenge, developing a scientifically literate citizenry requires training our students to think critically and develop an understanding of how evidence is used to construct, support, and evaluate claims (Luykx & Lee, 2007;Sampson & Clark, 2008).It also requires supporting students in developing the ability to apply sound statistical reasoning in their everyday life decisions (Franklin et al., 2007).
Both the Next Generation Science Standards (NGSS Lead States, 2013) and the Common Core Standards (Common Core State Standards Initiative, 2010) address the need for a more scientifically literate citizenry through an emphasis on practices that involve using or interpreting data such as constructing explanations, engaging in argument from evidence, and evaluating information.However, studies in both science and mathematics education have shown that students often struggle with skills related to these practices, including understanding variability in data (Ben-Zvi, 2004;delMas & Liu, 2005;Reading, 2004), interpreting and using statistical information (Garfield & Ben-Zvi, 2004), reasoning based on evidence (Driver, Newton, & Osborn, 2000), and extracting pertinent information from graphical representations (Bieda & Nathan, 2006;Friel, Cursio, & Bright, 2001).People's understandings of variability and its implications are of particular concern to educators because these skills are a foundational aspect of statistical reasoning and are essential for understanding the degrees of certainty with which claims can be made (Zawojewski & Shaughnessy, 2000).In a very direct way, the difficulties students experience Irish et al. / Data Explorations 2 / 15 with these skills may hinder their progress toward becoming the environmentally literate citizens and critical consumers of information that today's environmental problems require.
The Data Explorations in Ecology Project (DEEP) was a professional development effort designed to address environmental literacy issues in secondary science classrooms through the creation and implementation of curricular modules designed to support students in gaining proficiency with a variety of data exploration practices.These modules engaged students in place-based, evidence driven, inquiry activities that involved the use of both first and second hand data to explore locally relevant environmental issues.The data exploration practices focused on included a variety of skills essential to the practices of constructing, defending, and evaluating evidence-based claims.These skills include, but are not limited to, (1) interpreting data representations; (2) identifying variability in a data set; (3) identifying potential sources of variability; (4) understanding the implications of variability in a data set; (5) understanding how variability in data can influence claims based on those data; and (6) understanding how to evaluate and critique claims based on the available evidence.The focus of this study is on understanding students' proficiency with these skills across grade levels.We were also interested in gaining insight into students' interest in, and attitudes toward, data exploration practices.Our hypothesis was that explicit instruction and practice with data exploration practices would not only increase students' interest and proficiency with these skills, but also result in students' increased ability to construct, defend, and evaluate claims based on data.The specific questions addressed in this study include: 1. What do secondary students know, and what are they able to do, in terms of data exploration practices before and after participation in an environmental science unit focused on data exploration?
2. How do students' knowledge, skills, interests and attitudes towards data exploration practices compare across grade levels in secondary schools?

CONCEPTUAL FRAMEWORK
This study was designed to answer questions about middle and high school students' proficiency with, and interest in, a range of data exploration practices.Data exploration, in this study, is defined as the process in which individuals work with data sets to uncover patterns, characteristics, and points of interest inherent in the data using a wide variety of data summarizing, data visualizing, and data interpretation practices.This study operates from the perspective that an understanding of data exploration practices is an important aspect of environmental literacy because of the role these practices play in the processes of forming and evaluating claims.In science, an understanding of how data were collected, analyzed, and reported on is essential to evaluating claims made based on the data.These understandings allow individuals to make more sound judgments about the quality of evidence used to support and claim and the levels of certainly that can be reasonably attributed to a claim.Engagement in these practices can also contribute to environmental literacy by facilitating content knowledge learning.In the vision of the NGSS, students foster progressively deeper understandings of the content and core ideas of science disciplines by actively engaging in the scientific practices of those disciplines (NRC, 2012).
The findings presented here are organized using a framework we have developed for describing the variety of data exploration practices required throughout inquiry and critique processes (Figure 1).This approach is designed to help reveal insights about the knowledge, skills and attitudes students have in relation to the different practices required throughout complete inquiries.Five interrelated aspects, or phases, of inquiry and critique processes organize our framework.These include (1) a study design and implementations phase, (2) a data manipulation phase, (3) a phase that involves data analysis, summary and visualization, (4) a filter and synthesis phase, and (5) a communication and application phase.Each of these different phases of inquiry and critique processes involve a different set of data exploration practices.Our framework describes these phases and the data exploration practices they entail, not as isolated activities, but rather as interrelated practices that each inform and contribute to each other.Part of the usefulness of this framework is in making explicit where each of these data exploration practices fit within the context of inquiry and critique processes, how the practices employed at each phase can contribute to

Contribution of this paper to the literature
• This study provides unique insight into secondary students' knowledge and skills with a variety of data exploration practices across grade levels and evaluates how students use these skills in the creation and evaluation of data-driven claims about the environment.
• This study provides unique insight into secondary students' interest in, and attitudes toward, a variety of data exploration practices.
• This study introduces a conceptual framework for making explicit where each data exploration practice fits within the context of inquiry and critique processes.This framework is designed to support both students and teachers as they construct or critique explanations.

EURASIA J Math Sci and Tech Ed
3 / 15 the other phases, and how the data exploration practices required during inquiries are directly connected to the data exploration practices required for critique.In doing so, this framework acts as a support for both students and teachers as they develop inquiries and as they construct or critique explanations.

DEEP Curriculum Modules
The curriculum modules developed during this project were co-constructed by teachers participating in the DEEP professional development workshops, education specialists, and scientists from a local area research institute.The teachers were participants in a year-long PD program during which they attended a series of workshops that provided them with opportunities to engage in initial versions of the DEEP curriculum modules themselves and develop proficiency with a series of teaching practices essential to the implementation of the modules (Table 1).As part of the PD program, the teachers also worked in professional learning teams (PLT) to revise at least one of the modules to best match the context they were teaching in and then implement it with their own students.The revisions made were designed to tailor the modules to account for each teacher's individual teaching contexts including the data collection opportunities available, the subject being taught, the grade level of the course, and the ability level of the students.Each PLT included at least one teacher and one education specialist.Each of the teams was also provided with multiple opportunities to consult with scientists from the research institute.The modules developed included investigations into issues related to hydrofracking, salting roadways, invasive species, and stream acidity.Each module lasted between five to ten instructional days depending on the teachers' individual implementation.While each module, and teachers' implementation of the modules, was highly individualized, they all shared a focus on (1) using the teaching practices emphasized during the workshops; (2) targeting all five phases of the DEEP framework for inquiry and critique; and (3) the use of both first and second hand data to explore locally relevant environmental issues.

Participants
The participants for this study included students in the courses of each of 10 high school science teachers and 1 middle school science teacher.In total, data from 365 students, representing 11 different schools in the Mid-Atlantic region of the United States were included for analysis.For the purpose of this paper, the participants are divided into groups based on the grade level the course they were in was designed for.These groups included 7 th grade, 9 th grade, 10 th grade, and a combined grouping for students taking courses designed for 11 th and 12 th graders.The course titles, total number of teachers, and total number of course sections for each grade level are reported in Table 2.

Data
The data collection instruments used in this study included pre-tests, post-tests, focus group interviews, an article critique assignment, and a survey.These instruments were designed for the purposes of this study and were administered by teachers as part of the course assessments associated with the DEEP lesson module they implemented.The number of participants from each grade level course that completed each data collection instrument is provided in Table 3.Not all students completed all of the assessments, but all data for each grade level were drawn from the same group of participants.
The pretest, post-test and focus group interviews were designed to evaluate students' data exploration knowledge and skills.The pre-and post-tests were designed to target the same set of knowledge and skills before and after engagement in the DEEP lesson modules, but each used different contexts and data representations to do EURASIA J Math Sci and Tech Ed 5 / 15 so.The items on the pre-test used the issue of hydrofracking as a context for the questions and used representations of river water temperature data.The items on the post-test used the issue of road salt as a context for the questions and used representations of river water salinity data.On each test, a combination of fixed response and open response items were included.In terms of participants, only students who completed both the pre-test and the posttest are included in the numbers reported.
The focus group interviews were designed to gain additional insight into students' data exploration knowledge and skills.These interviews were conducted with groups of 4-6 students chosen randomly by the teachers and were conducted within one week of completion of a DEEP module.During the interviews, students were asked to individually complete items on a worksheet assessing their data exploration knowledge and skills and then, as a group, were asked questions about why they answered the items the way they did.Student responses on the individual worksheets and during group discussions were both included as data.Students in the 7 th grade courses did not participate in the focus group interviews.
The article critique assignment was designed to evaluate students' ability to apply their data exploration knowledge and skills to the critical evaluation of an article that provides data to support a claim about an environmental issue.Students were provided with a list of articles on either the issue of hydrofracking or the use of salt on roadways and were asked to choose one article from the list to critically evaluate.The article evaluation was submitted in the form of an essay in which students were asked to identify the claim, evidence, and reasoning provided in the article, provide criticism of the argument based on the evidence provided, and then propose new research, using new or existing data, that would better address the question posed in the article.Each article included was intentionally chosen based on the presence of critique-able elements.Students in the 7 th grade courses did not complete article critique assignments.
Finally, a survey was administered at the completion the data exploration unit.This survey was designed to gain insight into the students' interest and attitudes toward the data exploration practices they had been engaged in during the unit.The interest items asked students to rate their level of interest in each data exploration practice listed.The attitude items asked them to indicate the degree to which they agreed or disagreed with a series of statements about their experience in the data exploration unit.

Analysis
The analysis of the fixed-response items included in this study involved tallying the number of students who responded in a given way.Findings from these items are reported in terms of the percentage of students from each grade level that responded in a certain way.
The open-response items were analyzed using a multi-stage and iterative coding procedure, guided by the processes of first exploring and describing, then ordering and explaining suggested by Miles and Huberman (1994).After gathering all relevant data, we as researchers independently conducted a round of open coding to identify emergent patterns and themes for different items in the data.For each item analyzed, potential codes were discussed and evaluated based on their ability to usefully characterize pertinent results.Once codes were agreed upon, a second round of coding was conducted to apply the codes.The results of this round were compared across researchers to address inconsistencies.A third and final round of coding was conducted based on the refined criteria.Reliability among coders was established through a comparison of the results from the final round of coding.Inconsistencies were addressed through discussion and recoded.Although inter-rater reliability was not calculated for our coding procedure, the researchers were in close contact throughout the analysis process, discussing discrepancies that arose and modifying coding procedures to further clarify if needed.This iterative, qualitative process brought the researchers to agreement around the data analysis.

RESULTS
The findings presented in this section start by highlighting students' knowledge and skills in regard to a variety of data exploration practices including their ability to (1) interpret basic trends in data representations; (2) identify variability in data; (3) describe potential sources of variability in data; (4) describe the importance and implications of variability in data; and (5) evaluate claims based on the available evidence.Lastly, we provide the survey results on students' interest in, and attitudes toward, a variety of data exploration practices.

Interpreting Data Representations
The pre-test and the post-test each included one item designed to reveal insight into students' ability to interpret data representations.For the item on the pre-test, students were asked, "Looking at Graph 1 (Figure 2), how has the water temperature in the Hudson River changed from 1946 to 2006?"For the post-test item, students were asked, "Looking at Graph 1 (Figure 2), how has the salinity in the Hudson River changed from 1955 to 2010?"In Irish et al. / Data Explorations 6 / 15 responding to each item, students could choose from a list of six fixed response items including, "Increased", "Decreased", "Increased then decreased", "Decreased then increased", "No change", and "It's hard to tell".On the pretest, responses were considered correct if they indicated either "Increased" or "Decreased then increased".On the post-test, only the response, "Increased" was considered correct.
In comparing student performance on the pre-and post-tests across grade levels, two trends become evident.First, on the pretest, the percentage of students correctly interpreting the representation increased with grade level.Eighty-one percent of the 7 th grade participants, 89% of the 9 th grade participants, 94% of the 10 th grade participants, and 97% of the 11 th -12 th grade participants were able to correctly identify the trend in the data displayed in the graph.Secondly, a high percentage of students across all grade levels were able to correctly identify the trend in the graph on the post-test item, with 95% or more interpreting the trend in the graph correctly.
The focus group interviews also included an item evaluating students' ability to interpret basic trends in graph.During these interviews students were shown one of two versions of a graph displaying population density data over time for the same organism at two different sites and asked to write a claim on their answer sheets about the possible trend in the data from each site (Figure 3).While the data displayed and the questions asked were identical, the sites compared and the organisms counted were different for each version of the item.
In responding to these items, all 9 th and 10 th grade students, and 88% of the 11 th -12 th grade students wrote answers that accurately describe the trend displayed for the tick density data.For the mussel density data, 61% of the 9 th graders, 75% of the 10 th graders, and 79% of the 11 th -12 th graders were able to write an accurate description of the trend displayed.Because the data displayed were the same for each version, we find it interesting that the results of this interview differed depending on the context of the question.These results indicate that there may be

Understanding Variability in Data
Identifying variability.The pre-test, post-test, and focus group interviews each asked students to look at these same representations (Figures 2 and 3) and identify the period or site with the most variability in the data displayed.They were also asked to explain why they picked that period or site as having the most variability.Students were considered correct if they responded that "Period A" showed the most variability for the pre-and post-test items.For the focus group interviews, they were considered correct if they indicated South Troy or Millbrook.Student responses to the question of why they picked that period were analyzed and described as either reflecting an accurate description of the variability present in the data representation or an inaccurate description.
On the pretest item, the percentage of students that correctly identified Period A as showing the most variability increased with grade level.Sixty-two percent of the 7 th grade participants, 83% of the 9 th grade participants, 90% of the 10 th grade participants, and 97% of the 11 th -12 th grade participants provided correct responses (Table 4).Performance on the post-test item reflected a similar trend in that 57% of the 7 th graders, 78% of 9 th graders, 94% of the 10 th graders, and 98% of the 11 th and 12 th graders provided correct responses.However, in comparing pre-and post-test performance, fewer 7 th and 9 th grade participants provided correct answers on the post-test item as compared to the pre-test item.Five percent fewer students provided correct answers in each case.During the focus group interviews, all students across each grade level were able to correctly identify the site with the most variability.
Describing and defining variability.In responding to the prompt on the pretest asking them to explain why they picked that period as having the most variability, 41% of the 7 th graders, 61% of the 9 th graders, 67% of the 10 th graders, and 89% of the 11 th and 12 th graders provided an accurate description of the variability present in the data representation (Table 4).For example, on the pre-test, one 9 th grade student wrote, "Period A's temperatures were spread out over the widest range of temperatures".On the post-test item, an additional 3% of students in the 7 th grade courses and 14% of students in 10 th grade courses provided accurate descriptions.The number of students in the 9 th and 11 th -12 th grade courses providing correct answers did not change pre-to post-test.
In responding to the focus group interview question asking them to provide a definition for variability, 61% of the 9 th graders, 75% of the 10 th graders, and 79% of the 11 th and 12 th graders were able to provide a definition that was deemed accurate.Examples of accurate definitions of variability provided by students during these interviews are provided in Table 5.
Identify potential causes of variability.Both the pre-test and post-test included items asking students to list at least two possible causes of the variability in temperature/salinity within any given time period.In responding to this prompt on the pre-test, 73% of the 7 th graders, 74% of the 9 th graders, 71% of the students in the 10 th grade A variety in data when not everything is the same.
Variability is when something is constantly changing and doesn't stay the same for too long.
The difference between something, how much one thing is different from or varies from another.
11 th -12 th The deviation from the average of a set of values; how different the data points are.How different data points in a set are from one another.
The possible difference between all of the data collected over a period of time.
courses, and 94% of the students in 11 th and 12 th grade courses were able to list 2 plausible causes.On the post-test item, 15% more of the students in the 10 th grade courses were able to list 2 plausible causes.However, the percentage of 7 th graders providing 2 plausible causes decreased by 18% pre-to post-test, students in 9 th grade courses decreased by 11%, and students in 11 th -12 th grade courses decreased by 7%.These results are displayed in Table 6.
Importance of considering variability.The pre-test and the post-test each included an item asking students to explain why they think it is important to think about variability in a set of data.Student responses to this item were evaluated and characterized as either reflecting or not reflecting the importance of variability.Examples of answers in each category are provided in Table 7.On the pre-test, 48% of the 7 th graders, 51% of the 9 th graders, 77% of the 10 th graders, and 79% of the 11 th -12 th graders provided statements that reflected the importance of considering variability in a set of data.On the post-test, these numbers increased by 9% for 7 th , 9 th , and 11 th -12 th graders, and 7% for 10 th graders.Across all grades, and both tests, the most common statements reflecting the importance of considering variability in a set of data include descriptions of variability in data as providing evidence of natural variations in a system, as influencing the confidence of a claim, and as an indication of potential measurement error.

Evaluating a Claim Based on Evidence
The pre-test, the post-test, and the article critique assignment each included items requesting that students evaluate a claim based on the available evidence.In the case of the pre-and post-test items, students were asked to evaluate a claim that is supported using a representation showing a subset of a larger data set (what we call "cherrypicked" data).Students were provided with representations of both data sets and asked if they agree or disagree with the claim and to explain why they agree or disagree (see Figures 4 and 5).For these items, our interest was not in whether students agreed or disagreed with the claim, but rather in whether or not they addressed the data provided in explaining their reasoning for their stance.In the case of the pre-test item, only 2% of the 7 th graders, 13% of the 9 th graders, 19% of the 10 th graders, and 37% of the 11 th -12 th graders cited data in their response.On the post-test, the number of students citing data in their response increased across all grade levels to 14% for 7 th graders, 25% for 9 th graders, 39% for students in 10 th grade courses, and 67% for students in 11 th -12 th grade courses.These results are provided in Table 8.EURASIA J Math Sci and Tech Ed

/ 15
On both the pre-and post-tests, the overall trend in the data displayed was the aspect of the data most frequently cited by students in support of their claim.Two percent of the 7 th graders, 6% of the 9 th graders, 14% of the 10 th graders, and 31% of the 11 th -12 th graders mentioned the trend in support of their claim on the pre-test (Table 8).On the post-test these numbers increased, with 11% of the 7 th graders, 16% of the 9 th graders, 23% of the 10 th graders, and 48% of the 11 th -12 th graders mentioning the overall trend in the data as a means of supporting their claim.Other aspects of the data cited by students in support of their claim include the differences in scale between the two representations provided and the variability in the data displayed.Some students mentioned more than one aspect of the data to support their claim.
For the article critique assignment, students were asked to provide criticism of the argument presented in a persuasive article about an environmental issue.This assignment was designed to evaluate students' ability to apply their data exploration knowledge and skills to the critical evaluation of arguments that cite data in support  of a claim.As was the case with the items on the pre-and post-tests, our interest was in identifying the ways in which students addressed the evidence provided in the article.In analyzing the critique essays produced by students in response to this assignment, we found that 9% of the 9 th graders, 16% of the 10 th graders, and 46% of the 11 th -12 th graders cited data in their critique.The most common critiques of the data provided in the article included source bias, an insufficient amount of data to support the claim being made, and assertions that the data provided is irrelevant to the claim (Table 9).Sixty-seven percent of the 9 th graders, 52% of the 10 th graders, and 54% of the 11 th -12 th graders offered no critique at all.Lastly, 24% of the 9 th graders, 32% of the 10 th graders, and 4% of the 11 th -12 th graders offered critiques based entirely on personal opinions or values toward the issue addressed in the article.For example, in critiquing one of the road salt articles that cites river salinity data to support the claim that we should limit our use of salt on roadways, one student in a 10 th grade course wrote, "I believe that the proper amount of salt should be applied to roads to keep them safe."While we recognize road safety as a critical consideration in this debate, we note that the student did not address the river salinity data in their critique and instead cited safety as their reason for supporting the use of road salt.

Student Interest in Data Exploration Practices
After engaging in the DEEP lessons, students were asked to complete a survey that included items about their level of interest in the various data exploration practices they had been engaged in during the lessons.These items asked students to rate their interest in each of the practices listed on a scale of 0-4 (0 = Not at all; 1 = just a little; 2 = somewhat; 3 = a lot; 4 = extremely interested).The findings from this survey indicate that, regardless of practice, students across grade levels have more interest in working with their own data than they do in working with data from other sources (Table 10).A greater number of students indicated "a lot", or "extreme" interest in the practices of collecting data, graphing data, making sense of data, using data to answer a question, make a conclusion, or make a prediction when working with their own data over working with data from other sources.The one exception to this trend was 9 th grade students' interest in gathering data.Twenty-three percent of the students surveyed indicated "a lot" or "extreme" interest in collecting their own data and 29% indicated "a lot" or "extreme" interest in gathering data from other people.

Student Attitudes Toward Data Exploration Practices
The survey administered at the end of the DEEP lesson modules also included items assessing students' attitudes toward data exploration practices in general.These items asked students to rate the extent to which they  agreed or disagreed with a series of statements about data exploration practices using a scale of 1-5 (1 = strongly disagree; 2 = disagree; 3 = neutral; 4 = agree; 5 = strongly agree).Findings from this aspect of the survey indicate that at least some students appreciate that there are benefits to learning about data exploration practices (Table 11).
The items that had the most "agree" or "strongly agree" responses included those that described data exploration practices in positive terms, including, (1) The lessons made more sense because we worked with real data that we collected or scientists collected; (2) Working with data improved my ability to evaluate whether someone's claim is true or not; and (3) Working with data improved my ability to answer a question.There did not seem to be any distinguishable patterns in responses to these items across grade levels.
Students' appreciation for the benefits of engaging in lessons focused on data exploration practices is also shown through their responses to the statements about the drawbacks of working with data (Table 11).The items that had the least "agree" or "strongly agree" responses included those that described data exploration practices in negative terms, including, (1) The lessons were harder to understand because we used data; (2) Working with data is not important for learning about or doing science; and (3) Working with data did not improve my ability to understand or do science.The only distinguishable pattern in responses to these items across grade levels is that the 10 th grade students provided the fewest "agree" or "strongly agree" responses for each of these items.

DISCUSSION
This study was designed to answer questions about middle and high school students' proficiency with a range of data exploration practices including (1) interpreting data representations; (2) identifying variability in a data set; (3) identifying potential sources of variability; (4) understanding the implications of variability in a data set; (5) understanding how variability in data can influence claims based on those data; and (6) understanding how to evaluate and critique claims based on the available evidence.We were particularly interested in understanding students' proficiency with each of these practices across grade levels.This study was also designed to provide insight into students' interest in, and attitudes toward, data exploration practices.We hypothesized that explicit instruction and practice with data exploration practices would not only increase students' interest and proficiency with these skills, but also result in students' increased ability to construct, defend, and evaluate claims based on data.The findings show that in the case of each of the practices examined, there are some students that can demonstrate proficiency.Even before engaging in the DEEP modules some students were able to provide correct answers to items on the pre-test.The findings also indicate that, for the most part, students progress in their understandings of, and ability to perform each of these practices as they advance through school.In the case of almost every item analyzed, the number of students answering an item correctly increased as grade level increased.However, the findings provide only limited evidence in support of our hypothesis that student proficiency with these skills would increase as a result of explicit instruction.For most items, and across grade levels, there were only slight increases in the number of correct answers on the post-test as compared to the pre-test.There were also some instances in which fewer students answered an item correctly on the post-test compared to the pre-test.There are a couple of factors that could potentially explain these results.One is that engagement in the modules was not enough to facilitate an increase in learning of the data exploration practices targeted.If this is the case, then considerable modifications to the modules are going to be required for them to be considered for future classroom use.The other factor that could explain these results is that responded fatigue played a role in how the students approached the evaluation tasks.Respondent fatigue occurs when participants become tired of the tasks required by the study and the quality of the data they provide begins to deteriorate.We saw some evidence of this phenomenon in the number of items left unanswered on the post-test.Across grade levels, many more students chose not to respond to items on the post-test as compared to the pre-test.
Despite students' proficiency with some data exploration practices, and the apparent progress students demonstrate as they advance through school, there remain areas of significant concern in terms of student's data exploration knowledge and skills.Analysis of the data indicates that, in general, students know what variability is, are able to identify it, and are able list potential sources of it.However, the data also indicate that this understanding is limited.Even in the upper grades, many students are not yet able to explain why variability is important for answering a scientific question, making a claim, or formulating a prediction.Previous efforts investigating students' understanding of variability have reported that most students face difficulties in understanding the concept of variability, in part, because it is a complicated domain to teach and learn (Ben-Zvi & Garfield, 2004) and that even after explicit instruction, many still harbor misconceptions in reasoning about variability (Chan & Ismail, 2013;delMas & Liu, 2005;Lee & Meletiou-Mavrothesis, 2003).Studies focusing on pre-service and practicing teachers have indicated that the difficulties people face in understanding and making use of this knowledge persist beyond secondary school and that college students and teachers also struggle with concepts related to variability in data and the implications of variability in data (Confrey & Makar, 2002;Hammerman & Rubin, 2004).
We believe the difficulties students have with understanding the implications of variability in data directly contribute to their environmental citizenship practices.Reasoning about variability and the implications of variability have been identified as critical aspects of statistical reasoning that directly contribute to an individual's ability to evaluate and critique claims based on data (Zawojewski & Shaughnessy, 2000).Because students are exposed to a plethora of environmental claims and decisions on a daily basis, their ability to make sense of the variability in the data presented to support an argument becomes a critical concern.In a very direct way, the difficulties students experience with these skills may hinder their progress toward becoming the environmentally literate citizens and critical consumers of information that today's environmental problems require.Our findings provide some evidence of this connection.When presented with claims about the environment based on variable, or limited data, many of the participants in this study neglected to address these issues and instead chose to base the reasoning for their stance on something other than the data provided.
These finding resonate with previous research indicating that learning how to engage in productive scientific argumentation to justify or evaluate explanations based on data is difficult for students (NRC, 2012).Some researchers have proposed that to address these difficulties, students need to learn more about the types of claims that scientists make, how scientists advance them, what kinds of evidence are needed to warrant one idea over another, and how that evidence can be gathered and interpreted (Kelly & Chen, 1999;Osborne, 2002;Sandoval & Reiser, 2004).We assert that our conceptual framework can be useful in this endeavor.The DEEP conceptual framework for inquiry and critique explicitly situates data exploration practices in a larger framework characterizing and distinguishing critical stages of inquiry processes, thus helping track how different practices manifest at different stages (Figure 1).When scientists conduct research and students do inquiry, they move along a continuum from raw data to arguments, integrating analysis of their own data and consideration of others' data and representations along the way.Our framework is designed to help teachers and their students understand evidence-based inquiry and facilitate recognition of which parts of inquiry they were focusing on in their classrooms.It is also designed to make explicit that different practices and standards apply for different phases and that while each phase has distinct practices, meaningful progress in completing the phase either for inquiry or critique requires grounding in the other phases.For example, critique of a graphical representation without an understanding of the data collection and manipulations that went into it, nor the claims it might be used to address, is a shallow, mechanical exercise of limited value.The DEEP Framework incorporates key ideas from the NGSS while providing a visual of both inquiry and critique in the scientific process.It also helps the learner understand that at each step of inquiry, there are associated inquiry and critique practices which are essential for the successful understanding of science (Berland & McNeill, 2010;Ford, 2008).Such an approach is supported by the NGSS (NGSS Lead States, 2013) and its companion publication, A Framework for K-12 Science Education (NRC, 2012), which emphasize that science is a creative process through which arguments are constructed over time based on multiple types and lines of evidence.The Framework states, "As all ideas in science are evaluated against alternative explanations and compared with evidence, acceptance of an explanation is ultimately an assessment of what data are reliable and relevant in a decision about which explanation is the most satisfactory.Thus, knowing why the wrong answer is wrong can help secure a deeper and stronger understanding of why the right answer is right.Engaging in argumentation from evidence about an explanation supports students' understanding of the reasons and empirical evidence for that explanation, demonstrating that science is a body of knowledge rooted in evidence."

/ 15
We believe the DEEP conceptual framework for inquiry and critique can help students develop their ability to evaluate and critique claims based on evidence by providing a framework to make explicit the distinct data practices that occur at each key stage of inquiry and critique processes.It also makes explicit how the data decisions made at each phase of inquiry and critique processes contribute to the construction of an explanation.In doing so, the framework highlights key aspects of data exploration and reporting processes that students might consider when evaluating an explanation.In terms of the DEEP curricular modules, one potential implication of our study is to include more explicit connections between data exploration practices, the content and core ideas of science, and the practices of forming and evaluating claims based on evidence.Very few students cited data issues in the article critique assignments despite demonstrating some knowledge of these practices on other measures.Perhaps instruction that makes even more explicit connections between these related aspects of science could help address this issue.
We believe the findings from the survey evaluating students' interest in, and attitudes toward data exploration practices also support the usefulness of our framework.Despite the difficulties secondary students have with some data exploration practices, the results of the survey indicate some still enjoy and appreciate the usefulness of working with data.In particular, students indicated that they are interested in working with their own data over data collected by other people.Some of the students also indicated that they agreed or strongly agreed that the lessons focused on data made more sense, improved their ability to evaluate whether a claim is true or not, and improved their ability to answer a question.These findings indicate that some of the students in this study enjoyed and appreciated the usefulness of lessons focused on data exploration practices.However, our findings also indicate that many of the students had difficulty applying their data exploration knowledge and skills in the citizenship based contexts in which they were the most applicable.Previous studies have found that people often make decisions quickly and with very little thought when it comes to environmental issues, relying mostly on heuristics that limit and frame their choices (Kahneman & Tversky, 2000) and personal and community values (Gunckel, Covitt & Anderson, 2009).We saw some evidence of this tendency in the form of students who cited personal values instead of the data in their evaluation and critique of the articles included in the article critique assignment.We may also have seen evidence of this in the results of the focus group interview question that produced different results depending on the context of the question.In this instance, the students may have been more concerned with the issue of increasing tick numbers as opposed to an increase in mussels because of the prevalence of Lyme disease in the region in which this study was conducted.We believe the DEEP framework can help combat these tendencies by reminding students of the plethora of data issues they might seek to address in their evaluation and critique of explanations that provide data in support of a claim.We also assert that this increased data literacy will result in more informed, data driven decision-making about environmental issues.

LIMITATIONS
Several factors should be considered when interpreting the findings from this study.To begin with, the preand post-tests were different.Corresponding items were designed to measure the same set of knowledge and skills, but no testing was conducted to verify the equivalence of these items.Because of this, comparisons between student performances on the pre-and post-tests are somewhat limited.We also did not conduct any significance testing to evaluate differences between pre-test and post-test performance by individual student or across grade levels.Because of this, we are unable to report on whether or not the differences described in either measure are statistically significant or not.Lastly, respondent fatigue may have played a part in the data collection process as many students failed to complete both the pre-and the post-tests.

CONCLUSION
This study contributes to the field of science education by highlighting students' abilities with a variety of data exploration practices across grade levels and by introducing a framework for characterizing critical stages of databased inquiry and critique practices, thus helping us track how different skills manifest themselves at different stages and across different age groups.This information can be used to facilitate greater levels of scientific literacy and environmental citizenship through the design instruction that effectively targets skills that contribute to students' ability to construct, defend, and evaluate evidence-based claims, which will in turn contribute to student's informed participation in environmental decision making and stewardship.

Figure 1 .
Figure 1.DEEP framework for inquiry and critique.This figure illustrates some of the data exploration activities that occur at different phases of inquiry and critique processes

Figure 2 .
Figure 2. Data representations used on the pre-and post-test items designed to assess students' ability to interpret basic trends in a graph and identify variability

Figure 4 .
Figure 4. Pre-test item asking students to evaluate a claim based on the available evidence

Table 1 .
Teaching practices essential to the implementation of the lesson modules

Table 3 .
Summary of participants per data source

Table 4 .
Results for pre-and post-test items evaluating students' ability to identify and describe variability in data

Table 5 .
Examples of accurate definitions of variability provided by students during the focus group interview

Table 6 .
Results for pre-and post-test items evaluating students' ability to list two possible causes of variability in data

Table 8 .
Results for pre-and post-test items evaluating student's use of data to support

their claims Response 7 th 9 th 10 th 11 th -12 th Pre Post Pre Post Pre Post Pre Post
*Some students mentioned more than one aspect of the data to support their claim Irish et al. / Data Explorations 10 / 15

Table 9 .
Results for the article critique assignment evaluating students' use of data to critique claims Some students mentioned more than one aspect of the data in their critique *

Table 10 .
Students indicating a lot or extreme interest in data exploration practices on the survey

Table 11 .
Students indicating they agree or strongly agree with the statements listed on the survey