Understanding Factors related to Undergraduate Student Decision-making about a Complex Socio-scientific Issue: Mountain Lion Management

We outline an instructional strategy for supporting students’ science literacy skills using a structured decision-making tool in an interdisciplinary undergraduate course. Instructional tools support basing complex socioscientific issues decisions on a reasoned analysis of tradeoffs among multiple conflicting values rather than heuristics, such as social norms. We explored the factors related to students’ decision-making about mountain lion management by examining if students’ value orientations, identity, or knowledge predicted their management decisions before, during, and after engaging in structured decision-making where they performed a tradeoffs analysis. We found that student decision-making may align more closely to students’ value orientations and identity at the beginning of the course, suggesting that by the end of the course, students were less likely to make decisions centered in social norms and simplified single-value heuristics. A structured decision-making tool can be an effective way to support students’ examination of value tradeoffs when solving complex socioscientific issues.


INTRODUCTION
Many consider contributing to citizens' scientific literacy as the ultimate goal for science education.Science literacy is often defined as "the knowledge and understanding of scientific concepts and processes required for personal decision-making, participation in civic and cultural affairs, and economic productivity" (National Research Council, 1996, p. 22).Science classrooms that focus on decision-making around complex personal or civic decisions offer an opportunity for students to practice the knowledge and skills of a scientifically literate person, but more work needs to be done to operationalize science literacy as a science classroom instructional approach and learning outcome.Feinstein (2011Feinstein ( , 2015) ) and Feinstein, Allen, & Jenkins (2013) likewise focus science literacy goals on student practices that allow them to connect classroom science knowledge to their lived experiences in significant and satisfying ways.However, in contrast, researchers find that people overwhelmingly make decisions about complex issues based on cultural identity and approval of peer groups (Hart & Nisbet 2012;Kahan, Peters, Wittlin, Slovic, Ouellette, Braman, & Mandel, 2012).Students need to know when and how to apply science content knowledge to their daily decision-making and have opportunities to practice these skills, including debating and discussing complex issues that involve multiple stakeholders who have a suite of personal values, knowledge, and unique perspectives on economics, politics, social issues, and ethics (Feinstein, 2011;Kolstø, 2000).
Science classrooms that focus on decision-making around SSIs can provide opportunities for students to practice these science literacy skills and have the potential to create transformative models for science literacy instruction (Sadler, 2011;Zeidler, 2014).SSI instruction is situated in a real-world context, allowing students to navigate complex intersections between science, ethical considerations, and emotions (Sadler, 2011;Zeidler et al., 2005; Alred & Dauer / Factors of undergraduate decision-making 2 / 21 Zeidler, 2014).The multifaceted nature of ill-structured SSIs makes it difficult for students and instructors to disentangle values, emotions, and reason in the classroom, both in terms of appropriate instruction about the role of students' values and emotions, and in terms of understanding student progress towards a more scientifically literate mind-set around solutions to SSIs (Sadler, 2004;Sadler & Zeidler, 2005).Therefore, there is a need for models to explicitly teach decision-making in science classrooms in the context of SSIs in a way that allows instructors and students to clarify the values that are the source of emotions, acknowledge the important role of ethics and values, and integrate these ideas with scientific information.Having a structure for students' reasoning may support students' ability to understand what is objective and what is not, and promote progress in their own decisionmaking, idea communication, and consensus building.

Student Decision-making in Classroom Contexts
Employing a structured decision-making (SDM) process, based on normative models of decision-making, in the classroom can promote students' science literacy through group discourse about relevant, controversial societal issues (Aikenhead, 1989;Kolstø et al., 2006;Zeidler, Sadler, Simmons, & Howes, 2005).SDM may also support student practices such as community deliberation (Feinstein, 2011); recognizing cognitive biases inherent in human processes (Arvai, Campbell, Baird, & Rivers, 2004); applying conceptual understanding and detailed scientific information to problem solving; and applying epistemic and cultural understanding of the science process and the nature of information (American Association for the Advancement of Science, 1993, p. 4).
Despite an overall interest in decision-making, there are very few studies that have implemented and evaluated decision-making tools in a classroom.For example, a recent review of education research on students' decisionmaking by Fang et al. 2018 found only 24 studies, and just 13 of which were intervention studies.So, understanding how students approach decision-making about SSIs is limited.However, from this sparse empirical work, it is clear that the factors that play a role in students' decisions about SSIs in classroom contexts are extremely difficult to disentangle.As pointed out by Albe (2008), students are not mechanical, rational actors, and students' decisions are influenced by multiple factors, many of which are unknown or even appear to be random.Student decisionmaking may be driven by science knowledge, social norms, values and beliefs, personal experiences, or understanding of science epistemology (Albe, 2008;Grace & Ratcliffe, 2002;Hungerford & Volk, 1990;Kollmuss & Agyeman, 2002;Sadler, Chambers, & Zeidler, 2004;Zeidler, Walker, Ackett, & Simmons, 2002).Thus, despite some previous models proposed, there is a need for further exploration of classrooms tools to support student decisionmaking that explicitly connects to theory and is supported by on-the-ground research on student learning that begins to elucidate students' reasoning about SSI decisions (Ratcliffe, 1997;Fang et al., 2018).
The goal of this study is to begin to fill this literature gap by investigating how factors such as students' value orientations, identity group, or knowledge may explain students' decision-making in the context of a course using an SDM tool.The course was a required introductory undergraduate science literacy course for STEM and non-STEM majors that uses SSIs as a backdrop for a focus on student practices of decision-making and information literacy.We gathered data from students' decision-making work during a unit on the controversial issue of mountain lion hunting.

Contribution of this paper to the literature
• Provides a definition for quality decision-making, allowing for more clarity in instructional and research goals related to supporting students' socioscientific issue decision-making skills.
• Gives insight on the complex underlying basis for students' decision-making about a socioscientific issue through an analysis of the alignment between students' decisions and identity, value and knowledge variables.
• Provides evidence that using a structured decision-making framework in science classrooms could aid students' decision-making skills in regards to applying and analyzing value tradeoffs.

Supporting Students' Decision-making
To provide clarity to instructional and research goals, a "quality" decision must be defined.We rely on the decision-sciences to define a quality decision as one that depends on the quality of the process (Brewer et al., 2005;Wilson & Arvai, 2006) and displays the 1) ability of the decision-maker to understand technical information and apply it to the decision and 2) final choice reflecting priorities that result from evaluation of tradeoffs among the decision-maker's conflicting values (Wilson & Arvai, 2006).While some researchers in the science education community have discussed or researched the degree to which students' decisions are based in values versus information (Bell & Lederman 2003;Grace & Ratcliffe 2002;Kolstø 2000), we argue that a comparison of the two is a false dichotomy, and that both should play separate and important roles in the process.Further, informed decisions should be made based on ultimately obtaining what we value (Keeney, 1992), but with reasoned logic that weighs tradeoffs and is attentive to how scientific information predicts the performance of each choice (Gregory, Failing, Harstone, Long, McDaniels, & Ohlson, 2012).SDM tools can ease the burden of navigating the complexity of an issue, helping individuals make decisions that appropriately reflect what they prioritize (Wilson & Arvai, 2006).
In contrast, "value-judgments" represent an important example of a decision-making shortcut or heuristic that may lead to suboptimal decisions, and can be distinguished from values-based decisions.For example, making a purchase based on "what's the cheapest?"does not always result in optimal desired characteristics of the purchase.Wilson and Arvai (2006) and Wilson (2008) observed "value judgements" when they found that participants in experiments made decisions that reflected their initial affective impressions that aligned with identity-group and one specific value or concern, rather than focusing on previously stated conservation values intended for a valuetradeoffs decision-making process.
While it may not be realistic to expect individuals to use a SDM tool in their future lives for all complex decisions (e.g., before voting or making a medical decision), we propose that the experience of using a decision-making tool in a classroom setting is an opportunity to do objective problem solving and critical thinking, learn about psychological traps like cognitive biases and heuristics, exercise an understanding of the science process and the value of scientific information, and learn how decision support tools could provide a framework for decisionmaking, particularly in a community setting.In this study, we sought to gain some insight into if exposure to a structure decision-making process may guide individuals to become more thoughtful and deliberative in examining the application of value tradeoffs in their decision-making about SSIs.

Understanding the Basis of Students' Decisions
In order to assess increased quality of student decision-making, and in particular, students' ability to make decisions based on a reasoned analysis of tradeoffs and potentially recognize the role of cognitive biases in their own decision-making, instructors and researchers need a more complete understanding of the basis of student decisions.As mentioned earlier, students' decisions are based on complex factors (Albe, 2008), only one of which may be a systematic analysis of value-tradeoffs, which complicates how to pin-point students' thought processes in decision-making.When students give reasons for supporting a decision, they typically do not reveal the basis of their decision (e.g., it is rare to see students make a connection to in-groups or norms).Therefore, in this study we looked at correlative factors that may play a role in students' decision, including value orientations, identities of gender, rural background and hunting engagement, and ecology knowledge, and determine if these factors predict students' stances as a course progresses.
To explore a correlation between knowledge and students' stances, we measured student's knowledge specific to ecological concepts related to the mountain lion issue.Early research focused on the "deficit model," which suggests a direct relationship between scientific knowledge related to an issue, and peoples' attitudes that drive decisions (Sturgis & Allum, 2004;Allum et al., 2008.A sizeable body of empirical research shows that this perspective is over-simplistic in its assumptions about the nature of knowledge (National Academies of Sciences Engineering, & Medicine [NASEM], 2016).Yet, results of studies exploring this connection remain mixed, with some studies indicating that there is some relationship between knowledge and stances on issues contingent on particular factors (Allum et al., 2008;Ho et al., 2008;Brossard et al., 2009;Cacciatore et al., 2012).Therefore, different levels of knowledge sophistication closely related to concepts needed to understand the mountain lion issue may have an influence student stances.
In order to investigate the role of students' values and identities in decision-making, we look to theoretical frameworks from behavioral sciences.Value-Belief-Norm (VBN) Theory has been widely used to understand environmentally significant decision-making and predict behaviors relevant to diverse contexts and across cultures (Huffman, Van Der Werff, Henning, & Watrous-Rodriguez, 2014;Sussman, Lavallee, & Gifford, 2016;Whitley, Takahahi, Zwickle, Besley, & Lertpratchya, 2016).According to VBN Theory, there is a causal chain that moves from relatively stable, central elements of values, to beliefs and personal norms, and then to behavior (Stern, 2000).The value orientations we used are based on the Value-Belief-Norm (VBN) Theory (Stern, 2000) where individuals are aligned along three sets of beliefs regarding human-nature relationships (i.e., egoistic, altruistic, biospheric).While values are stable guiding principles in one's life of which there are few across cultures (Schwartz, 1992), value orientations are dimensions upon which related values cluster and predict general beliefs (De Groot & Steg, 2008).Value orientations can be a predictor of subjects' behavior; those with high biospheric value orientations were more likely, and those with egoistic value orientations were less likely, to engage in pro-environmental behavior (De Groot & Steg, 2008).Because the mountain lion issue deals primarily with tension between biospheric (wildlife conservation) and economic (ranching) issues, we were particularly interested in students' degree of polarity between biospheric and egoistic value orientations as a predictor of their decision or stance.
We used VBN theory to begin to understand if value tradeoffs informed the basis of students' decisions.Our reasoning follows the assumption that if students' identity and biopheric-egoistic polarity in value orientations are highly predictive of their decisions, then the students' decision may be based on social norms of an identity group, or a single-value judgement heuristic.Conversely, if students' decisions are not predicted by identity and biophericegoistic polarity in value orientations, then this may indicate that students are making a decision that reflects priorities among multiple values and their tradeoffs.In SDM, the student has an opportunity to consider personal values and weigh tradeoffs among decision outcomes that they care about.As a result, the causal chain between values, beliefs, and norms may be somewhat altered by individuals' reasoned analysis, so it is possible that biopheric-egoistic polarity in value orientations become a weaker predictor of decisions as a result of a SDM tool that supports analyzing tradeoffs among multiple potentially conflicting values.
Our approach included asking students their opinions about what we should do about the SSI before and after the course in an open-ended format.We considered this assessment as our unstructured mode of evaluating students' decision-making.We also investigated students' decisions that immediately followed their analysis using the SDM steps that requires a student to engage in a tradeoffs analysis and make a decision based on the analysis (as described below), which we considered our structured mode of evaluating students' decision-making.With the additional data we collected on students' value orientations, identities and ecological knowledge, we investigated what factors may predict or explain students' decisions about the SSI before, during, and after using a decisionmaking tool.While this approach has limitations (see discussion), determining the process that students are using to make decisions is extremely complex, and we believe this is a valuable first approach that will provide initial insight into the phenomenon, as well as information about the success of a course that asks students to use a SDM process.

RESEARCH QUESTIONS
This study explores the relationships between complex factors that may play a role in decision-making with regard to the conservation of a semi-isolated mountain lion population in the Midwestern United States.Additionally, this study seeks to understand the effectiveness of a course focused on decision-making practices about SSIs in its ability to support students in considering tradeoffs among multiple values related to the SSI.Our research questions are: 1. What are students' decisions about mountain lion management, and how did these decisions change throughout a course using a SDM tool?
2. Do factors (value orientations, identities or ecology knowledge) predict students' mountain lion management decision before, during or after using a SDM tool?
Understanding how students' decisions were aligned with expected social norms related to their identities or values can reveal how effective the SDM tools were in aiding students with engaging in systematic application of conflicting value tradeoffs to support their decision.Our hypotheses were that identities and polarity in biosphericegoistic value orientations would be less predictive, and ecology knowledge more predictive, of students' decision during SDM exercises and after the course than at the beginning of the course.Our ultimate goal was to provide instructors information about using SDM practices in the science classroom, as well as the potential for these tools to allow students to be reflective and deliberate in regards to their own values so as to engage in more effective decision-making.

Classroom SDM Tools to Support Value Tradeoffs Analysis
To explore factors that explain students' decision-making, we collected data from students enrolled in a required introductory undergraduate science literacy course for STEM and non-STEM majors, "Science and Decision-making for a Complex World," taught at a large university in Nebraska.A primary course objective was for students to distinguish between a) scientific information and b) criteria for solving the problem (connected to values, ethics, culture, economics, or politics).A second objective was for students to work with peers to use both scientific information and tradeoffs among valued outcomes to make a case for the best solution to important and complex SSIs.The course structure that asked students to apply an SDM tool to controversial SSIs salient to the geographic region in four two-week instructional units (see Dauer & Forbes, 2016 and Dauer, Sorensen and Jimenez in prep for more course description).The teaching approach we used aligns with our theoretical decision-making framework to support students' decision-making about complex SSIs, and included explicit instruction on decisionmaking practices and cognitive biases.We used an SDM tool based on normative models of decision-making (Hammond et al., 2015) as well as previous science education literature (Ratcliffe, 1997;Grace & Ratcliffe 2002;Grace, 2009;Lee & Grace 2010) meant to make students' decisions more explicit and to hypothetically reduce students' cognitive biases in decision-making (Dauer, Lute, & Straka, 2017).The steps of the tool were: 1. Define the Problem: What is the crux of the problem as you see it?2. Options: What are the options?(List the possible solutions to the problem.) 3. Criteria: How are you going to choose between these options?(Explain important considerations and what is valued in an outcome.) 4. Information: Do you have enough information about each option to evaluate based on your criteria?What scientific evidence is involved in this problem?What additional information do you need to help you make the decision?5. Analysis: Discuss each option weighed against the criteria.What are the trade-offs of each option?6. Choice: Which option do you choose? 7. Review: What do you think of the decision you have made?How could you improve the way you made the decision?
During the mountain lion unit, instruction was multidisciplinary, focused on understanding the scope of the in terms of ecology, economy, ethics, and society.Students worked through aspects of the steps of the decision-making process during lecture (steps 1-4), in small groups (step 4), and then as individuals they wrote responses for each step for the summative unit assessment.

Classroom Context: Mountain Lion Conservation
Wildlife conservation is a relevant, timely, and important SSI context for the classroom, especially given that declines in biodiversity were more rapid in the past 50 years than at any other time in human history (Sarukhan, et al. 2005).Because changes in biodiversity are tied to habitat loss associated with human development and food production, decision-making to conserve biodiversity is rife with trade-offs and controversy.In particular, conservation of a large predator species is an example of a SSI that provides an opportunity to weigh the complex needs and values of humans with strategies to protect the species in question.Additionally, conservation of large apex predator species can be controversial when people have fears about their safety or perceive a threat to their livelihoods (Kleiven, Bjerke, & Kaltenborn, 2004).
We developed an instructional unit around mountain lion (Puma concolor) conservation in Nebraska.While mountain lions are native to all of North America, humans extirpated mountain lions from Nebraska in the late 1800s, and no recorded observations of mountain lions occurred again until 1991.Most of Nebraska is unsuitable habitat for mountain lions to establish breeding populations, except for a few forested habitat patches in the northwest and north-central regions of the state.In 2013, genetic surveys of these regions indicated a small resident population of 19-22 individual mountain lions (Nebraska Game and Parks, 2013).The Nebraska Game and Parks Commission held the first managed hunting season for mountain lions in 2014.Controversy over management of this apex predator stems from a myriad of concerns related to human safety, livestock and pet protection, ecological biodiversity, genetic variability, economics, politics, and ethics.Because mountain lions are relatively new to Nebraska since their extirpation in the early 1900s, it is still unknown how mountain lions will impact the existing ecosystems, especially in a predominantly agricultural and rangeland landscape.

Variables that May Explain Student Stances on Mountain Lion Management
In addition to value orientations (described above), we examined identity factors that have been linked to attitudes and beliefs about predators and that may influence decision-making about mountain lion conservation.We also examined relevant ecology content knowledge.Below we briefly review these variables and how each may be important in understanding students' stance on this issue.

Gender
In a study on the public's concern about mountain lion attacks, gender was a predictive variable: Women expressed greater concern for mountain lion attacks than men, yet were less likely than men to support lethal control methods in residential areas (Zinn & Pierce, 2002).Additional studies show that men more strongly supported hunting of mountain lions than women (Thornton & Quinn, 2009;Zinn, Manfredo, & Vaske, 2000).

Rural background
There are few urban areas in Nebraska, all of which are in the eastern portion of the state where mountain lion encounters are infrequent.Therefore, rural students may be more likely to encounter mountain lions than urban students.Most livestock operations exist in rural areas, so students whose families own livestock may have more experience with mountain lions and therefore have more negative attitudes toward them because livestock are possible prey.One study on attitudes and beliefs toward mountain lions reported that rural residents actually had overall positive attitudes toward mountain lions compared to urban residents (Thornton & Quinn, 2009).They found that rural residents had more knowledge and experience with mountain lions, whereas urban residents had less experience, felt less informed, and thus expressed more fear of mountain lion attacks.Rural residents, however, may be more likely to support hunting mountain lions due to a myriad of social reasons including protest of a central political body pushing environmental regulation, general belief in anti-establishment rhetoric, and social disempowerment that rural communities sometimes experience (Wilson, 1997).

Stakeholder identity: hunting participation
In a study of public attitudes toward mountain lion recolonization in the Midwest (Davenport, Nielssen, & Mangun, 2010), hunting participation was the strongest predictive variable of people's management choice.Compared to non-hunters, hunters were more likely to support measures to regulate, rather than preserve, the mountain lion population.This most likely occurs because hunters tend to have more utilitarian wildlife value orientations where control over wildlife is important, whereas non-hunters are more likely to express protectionist values where "right to existence" is important (Davenport et al., 2010;Zinn & Pierce, 2002).

Ecology content knowledge
During the course, students had the opportunity to learn about food webs, trophic cascades, small populations and genetic variability all within the context of the mountain lion issue.However, it is unclear how people apply knowledge of ecological concepts to their opinions and decision-making, particularly in how prevalent alternative conceptions manifest themselves in the context of conservation decision-making.Students often fail to see the complex interconnectedness of food webs (Munson, 1994), which is important in the context of apex predator species that strongly impact trophic relationships.Another ecology concept requiring important consideration for wildlife decision-making processes is genetic variability and its role in population resilience to environmental change (Morrone, Katinas, & Crisci, 1996), which students often fail to recognize (Alred, Doherty, Hartley, Harris, & Dauer, 2019).Harvesting individuals from very small populations, such as in our case study, may have a large impact on genetic diversity.And, although it is possible for individual mountain lions from neighboring states to disperse through Nebraska's northwest corner where the breeding population currently resides, that area is reasonably isolated compared to most other mountain lion breeding areas in the United States.

Data Collection
In Fall 2015 we retrieved data from two lecture sections with 109 students in the first section and 114 students in the second section.The final sample size consisted of 110 students from the combined two lecture sections with the same instructor.We collected all data for the value orientations survey, identities, knowledge pretests, and posttests electronically through online survey software, Qualtrics.The decision-making tool was embedded within an end-of-unit assessment that students submitted to an online course management system for grading.We used both quantitative and qualitative methods to calculate student scores for each set of data.Scoring frameworks were EURASIA J Math Sci and Tech Ed 7 / 21 a mix of quantitative scoring, scoring based on existing coding frameworks, and new qualitative coding frameworks created as a result of this research.

Identity information
We collected self-report identity information for the following variables: major area of study, gender, whether students participate in hunting, hometown, and whether their hometown is rural or nonrural (urban or suburban).Students in this sample were representative of the course and were 60% STEM majors (Animal Science, Fisheries and Wildlife, and Environmental Science were top majors), 35% non-STEM majors (Hospitality, Restaurant, & Tourism Management; Agribusiness; and Agricultural Education were top majors), and 5% undeclared; 34% male and 66% female; 57% rural and 43% non-rural; and 45% hunter and 55% non-hunter.Students quoted in this study have pseudonyms.

Value orientations survey
The value orientations survey (De Groot & Steg, 2008) is a tool used to measure three sets of beliefs, or value orientations, regarding human nature: egoistic (concern for self), altruistic (concern for other humans), and biospheric (concern for all lives, human and non-human).The survey for the current study consisted of four items for each of the value orientations for twelve total value items.Students took this survey as part of a pre-course assessment only, rather than both pre and post, since values are normally stable (Stern, 2000) and because it was not our goal to change or track values.Students indicated on a 9-point Likert scale how important the twelve values were as "guiding principles" in their life.The scale ranged from -1 ("Opposed to my values"), 0 ("Not important") to 7 ("Extremely important").There was no correlation between biospheric and egoistic value orientation scores (R 2 =0.03).We calculated polarity between biospheric and egoistic value orientations as a subtraction between students' biospheric and egoistic value orientation scores to create a fourth "Bio-Ego" score as a composite score that incorporates both variables.

Ecological knowledge sophistication assessments and coding rubrics
To explore the relationship between students' level of ecological knowledge and decision-making about mountain lion conservation, we gave students a brief ecological knowledge assessment.After the mountain lion unit, students responded to a bonus quiz consisting of two multi-part questions, which served to assess their knowledge of food web complexity and their knowledge of small population-related concerns.Lectures relating to food webs, trophic cascades, genetic inbreeding and small populations took place before this knowledge assessment.We evaluated the ecological knowledge assessments separately for the two questions regarding food webs and small populations.Both questions were open-ended prompts that helped us gain insight into the range of sophistication in student responses.We developed coding frameworks based on previous work related to learning progressions in ecology (Alred et al., 2019;Hartley, Anderson, Berkowitz, Schramm, & Simon, 2011).
For the food web knowledge question, the focus was how well students could recognize ecological complexity in a real-world scenario (Terborgh et al., 2001) that had parallel concepts to the mountain lion situation.Students responded to a series of increasingly complex questions to identify direct and indirect connections between the predators, prey, and vegetation (Hartley et al., 2011; Appendix A).Students received a score of 0 to 5 (Appendix B).To simplify our statistical analysis, these five score categories were condensed into two categories, reflective of a lower score "0" for students scoring 1-3 indicating simpler, direct and indirect species relationships, or a higher score "1" for students scoring 4-5 indicating students who identified more complex, indirect connections in the food web model.
The purpose of the small populations knowledge question was to see if students expressed concern for genetic variability in small populations in a captive breeding setting (Alred et al., 2019).The open-ended question's context was that of an endangered species native to Nebraska.We asked students, "Imagine you are a wildlife biologist and you want to preserve one of our endangered predators in Nebraska, the swift fox, by starting a captive breeding program and reintroducing them into the wild. 1) How would you select a group of swift fox for your program, 2) How many swift foxes would you choose, and 3) Why?" A lower score reflected that students did not mention concern for genetic variability "0," and a higher score reflected that students included concern for genetic variability "1," including students who were both explicit and vague in expressing the concern.
Three independent researchers coded responses and reached an average 90% agreement for the food web knowledge scores and 72% agreement for the small populations knowledge scores.After resolving discrepancies, coders reached an average 95% agreement for food web knowledge scores and 90% agreement for small populations knowledge scores.

Students' decision in structured and unstructured contexts
Students' unstructured pretest and posttest decisions were collected in an ungraded, extra credit assignment before the start of the semester and during the final week of the 16-week semester.The students responded to a prompt that described the mountain lion management issue in Nebraska then asked, "What do you think should be done about this problem?Should we hunt mountain lions in Nebraska?Why should we do it/not do it?"(Appendix C).For the first two weeks of the semester, students learned about decision-making generally, then had two weeks of instruction and activity related to decision-making about mountain lion management.Individuals completed an SDM written assessment to respond to all 7 decision-making steps as the final summative assessment of the unit, which was due at the end of the fourth week.We analyzed students' structured "Choice" (Step 6) from their unit assessment (Appendix D).The SDM responses initially informed the inductive coding framework development for student decisions.We created a mountain lion management decision coding scheme to categorize the themes present in students' responses.Through the constant comparative method (Creswell, 2013), 4 specific theme categories emerged that reflected students' responses.
We then used this mountain lion management decision rubric for preliminary coding of the unstructured pretest and posttest decisions.After preliminary coding of the pretest and posttest, we modified the rubric to account for nuances in students' pretest and posttest decisions.Researchers re-coded the SDM responses to reflect the modified rubric.After this second round of coding, two independent coders reached 88% agreement for pretest, 96% agreement for SDM responses on the unit assessment, and 86% for posttest decisions.After resolving discrepancies, coders reached 100% agreement for pretest, 100% agreement SDM responses, and 99% for posttest decisions.

Data Analysis
Because the intent of this study was to explore the factors that may influence student decision-making in different contexts (unstructured before and after the class, and while using SDM), we performed three analysis, one for each context.Each dependent variable: 1) unstructured pretest decisions, 2) the management decisions ("Choices" Step 6) in the SDM tool on the unit assessment, and 3) the unstructured posttest decisions, were categorical (student decision) and nominal, therefore we chose multinomial logistic regression as a statistical test to determine which variables (value orientations, identities and ecology knowledge) predicted student decisions.Three multinomial logistic regressions for each context (pre unstructured, SDM in the unit assessment, post unstructured) were performed using SPSS Statistics for Macintosh (Released 2016, IBM Corp, Armonk, NY), each model had the same independent variables (value orientations, identities and ecology knowledge).One of the assumptions of multinomial logistic regression is independence of irrelevant alternatives, or the idea that eliminating some of the unchosen alternatives should not affect the selection of the option.Because the students' decision for mountain lion hunting was open-ended, and the categories were coded by researchers post hoc, we could not determine if we have violated this assumption of the model as we do not have information about what or how many alternative options the student considered.We used a Likelihood Ratio Test p-value of 0.1 to determine significance for the decision analyses based on precedence in the literature of similar research (Theobald & Freeman, 2014).For significant variables that predicted students' decisions, we calculated the odds ratio that a students' given status related to a given decision.

RESULTS AND DISCUSSION
To frame general patterns in student thinking about mountain lion management, we first describe the types of decisions that students offered and patterns of students' change in stance throughout the course (research question one), and we illustrate how these decisions changed throughout the course.We then discuss our analysis of variables that predicted students' decisions (research question two).

Decision Themes
We found four major themes in student decisions about how to solve the mountain lion issue.Themes ranged from the most extremely positive toward hunting to the most against hunting.The majority of students were prohunting (Figure 1).By the end of the course in the unstructured post-test responses, there was an increase in students in more moderate categories and a decrease in students in more extreme "pro" and "anti" hunting categories.

Yes: Hunt throughout entire state
Students in this category were in favor of hunting and did not explicitly exclude mountain lion breeding areas.Nebraska's current management plan does allow hunting statewide, but there is a hunting quota in regions of the state where a small population of breeding mountain lions exists.Responses often referenced other stipulations, such as a hunting quota, designated seasonal hunting seasons, or the use of dogs in certain areas of the state.For example, Colin said in his unit assessment, "This is the best option because it allows the mountain lions to continue living in the state, and slowly keeps their population under control in order to avoid young mountain lion wandering."Many students cited the need to control a growing population and protect people or livestock.Additionally, many students expressed an explicit or implicit desire to support mountain lion sustainability, often coupled with the idea of keeping the population numbers "under control.""Yes: Hunt throughout entire state" was the most popular management decision category with greater than 47% of students falling in this category throughout the course.A similar number of students fell in this category for both the pretest (n=65) and the SDM assessment (n=64), but the total decreased by 10 individuals for the posttest (n=52; Figure 1).

Yes: Hunt only outside of breeding areas
Students in this category determined that hunting could occur within the state but not within designated breeding areas.In Nebraska, breeding areas are determined based on the existence of a resident mountain lion population (i.e., evidence of a female mountain lion with kittens).Students in this category focused on population sustainability by protecting the small breeding population, yet allowing people to hunt solo males that disperse across the rest of the state where habitat is unsuitable for mountain lions.On her SDM assessment, Lily said, "Allowing for unlimited mountain lion hunting with a permit…and no hunting in the resident population of the Pine Ridge Unit would be the best choice…it would give the mountain lion population in the Pine Ridge Unit time for growth and development of a stable population…" For the pretest, no students fell in into this category; rather, it emerged starting in the SDM assessment during the course as students learned to distinguish between resident breeding populations and dispersing lions.This category contained the fewest number of students, comprising 13 of students in the SDM assessment, and 11 students in the posttest (Figure 1).

No: Do not hunt currently
Students in this category supported the idea that hunting should only occur if the mountain lion population becomes large enough for sustainable management or if the mountain lions become "overpopulated."For example, on the posttest, Marcus said, "I do not think that we should hunt mountain lions in Nebraska.I think that they are still a relatively small population and should only be hunted if they are posing a threat to livestock and human lives.We should make sure that their populations are stable enough before we start controlling their populations."Students learned about the concept of a sustainable population by examining a study that designated a sustainable harvest as equal to the intrinsic growth rate of a mountain lion population, which, is a maximum of 14 percent of the population (Beausoleil, Koehler, Maletzke, Kertson, & Wielgus, 2013).Nebraska's 2014 hunting quota of 4 total mountain lions may be considered over a "sustainable" amount for a population of approximately 22 individuals."No: Do not hunt currently" was the second-most popular management category on the SDM assessment (n=18) and posttest (n=35; Figure 1).

No: Do not hunt or kill
Students in this category expressed the idea that mountain lions should not be hunted or killed in any capacity.Students supported this sentiment with various reasoning, such as the population is too small or in terms of animal rights.Bonnie explained in her pretest, "From an animal lover perspective, humans have done nothing but inflict damage on the earth and particularly the animal kingdom… We often go in under the guise of 'controlling populations' and then we over hunt and overuse the resource...They have just as much of a right to be here as we do."Some students viewed mountain lion harvest as purely sport hunting rather than as a necessary management strategy.A few students thought mountain lions, if left alone, could positively impact the ecosystem by decreasing the deer population.The number of students in the decision category "No: Do not hunt or kill" declined over time (pretest, n=24; SDM assessment, n=15; posttest, n=12; Figure 1).

Variables Predicting Decisions
We examined students' value orientations (Table 1), identities and ecological knowledge (Tables 2 and 3) to better understand their relationship with the students' decisions and found differences between the pretest, SDM unit assessment, and posttest.
We found that some of the variables we measured predicted students' decisions in the pretest (multinomial logistic regression; p<0.001),SDM settings (multinomial logistic regression; p=0.074) and posttest (multinomial logistic regression; p<0.001).The broad patterns we observed in these variables between the four different management types were similar across all of our observation time points.For example, rural males who hunt were more likely to choose a pro-hunting management choice on the pretest, SDM assessment and posttest.Significant variables and their patterns are discussed below (Table 4), along with how these differences may reflect understanding of the students' decision-making processes.

Value orientations
Students' altruistic and biospheric value orientation scores were, on average, higher than their egoistic value orientation score (Table 1).The calculated Bio-Ego value orientations score predicted students' decisions in the pretest (p=0.051) and the SDM assessment (p=0.049),but not the posttest (Table 4).On all three assessment types, students with lower Bio-Ego value orientation difference mean scores were associated with more pro-hunting management strategies ("Yes: Hunt throughout entire state") and students with higher Bio-Ego value orientation difference mean scores were associated with non-lethal management strategies ("No: Do not hunt or kill") (Table 4).This relationship may be expected in that egoistic value orientations were a negative predictor and biospheric value orientations were a positive predictor of subjects' engagement in pro-environmental behavior (De Groot & Steg, 2008).
Value orientations may help clarify if we have met our classroom goal to support quality decision-making.A strong alignment between Bio-Ego scores and a stance on an issue may reflect that a student is engaging in a simple heuristic litmus test of alignment with a single prioritized value.In previous decision-making research in the context of the same course, value orientations predicted students' decisions on the pretest but not the posttest during a biofuels unit (Dauer et al., 2017), which the authors suggested was a reflection of students transferring decision-making skills learned on the SDM assessment to unstructured settings.Thus, we would consider weaker relationship between Bio-Ego scores and students decisions to be a sign of general classroom progress toward decisions that are more complex and nuanced than individuals' original stances, and potentially based on carefully evaluated tradeoffs among multiple values.
We found that value orientations were less predictive of students' decisions as the semester progressed.This may indicate that some students considered multiple values to a greater extent toward the end of the course, which aligns with our learning goal for students.However, the relationship between value orientations and students'  -------------------------------------------------------------- decisions were statistically significant on the SDM assessment, although the assessment was explicitly meant to support students' consideration of multiple values, so we expected the significant relationship to be lost during this assessment point.Therefore, the connection between the student practice of tradeoff analysis and value orientations as a predictor of students' decisions may be less straightforward than we expected, (and indeed, we discuss limitations using this approach, see "Limitations" section below), or the SDM assessment of the unit assessment may not be clearly scaffolding a trade-offs analysis.In future iterations of the course, asking students to assign weights to criteria and perform a semi-quantitative analysis of tradeoffs may make this practice more clearly defined on the unit assessments.

Identity factors
On all assessments, the general pattern was that rural males who hunt were more likely to be pro-mountain lion hunting than non-rural females who do not hunt.Gender was a significant variable on the pretest (p=0.029),but not on the unit assessment or posttest.To more efficiently view how gender relates to particular decisions (along with other identity variables), we calculated how much more likely males were to have a particular decision on all the assessments where a variable significantly predicted decisions.In particular, male students were 3 times more likely than female students to be in the "Yes: Hunt throughout entire state" category and female students were 4 times more likely than male students to be in the "No: Do not hunt currently" category.Rural background was a significant predictor of decisions on the pretest (p=0.011) and posttest (p=0.031)but not on the unit assessment (Table 4).Rural students were 3 times more likely to have a decision in the category "Yes: hunt throughout the entire state" on the pretest, while on the posttest nonrural background was a strong predictor; nonrural students were 8 times (pretest) and 19 times (posttest) more likely to make a decision in the "No: do not hunt or kill" category.Hunting identity significantly predicted students' decisions on the pretest (p=0.072;Table 4).On the pretest, hunter students were 3 times more likely than non-hunter students to be in the "Yes: Hunt throughout entire state" category and non-hunter students were 8 times more likely than hunter students to be in the "No: Do not hunt or kill" category.
These results suggest that identity variables, which may be strongly ingrained and influence what students value in their decision-making, were predictive of students' decisions at the beginning of the course (pretest) and to a lesser extent (only rural status was a predictor) on the posttest, both contexts without a decision-making structure.Students may be less likely to rely on value-based heuristics associated with their personal identity and social norms as a result of the course overall and when making decisions using SDM tools.

Ecological knowledge sophistication
Students' food web knowledge sophistication predicted their pretest decision (p=0.031) and SDM decision (p=0.031)but not the posttest decisions (Table 4).We had hypothesized that students with a higher understanding of food webs would better understand potential positive influences of resident apex predators on community health via trophic cascades.This understanding could translate into greater concern about extripation of mountain lions from state ecosystems expressed as two "No hunting" categories or the "Yes: Hunt only outside of breeding areas" category decisions.Our data somewhat supported this hypothesis.For example, students who had high sophistication food web knowledge more likely on the pretest (7x) and on the SDM assessment (<2x) to report a choice in the category "No: Do not hunt currently.They were also more likely to report a "Yes: Hunt only outside of breeding areas" category choice on the SDM assessment (4x) and pretest (<2x) than students who gave low sophistication responses.However, in contrast to our hypothesis, students who had a low sophistication food web knowledge score were more likely on the pretest (<2x) and the SDM assessment (4x) to report a "No: Do not hunt or kill" category choice than students who gave high sophistication responses.It is interesting that students with better understanding of food webs were more likely to give moderate decisions on this issue; however, we are unsure of how to explain this result.More research is needed to explore the connection between knowledge and its role in students' understanding and decision formation.
Small populations knowledge sophistication was predictive of students' posttest decisions (p=0.047) and not their structured decision or pretest decisions (Table 4).If students were concerned about the mountain lions' small population size, we would hypothesize that students with higher small populations knowledge would fall into the two "No hunting" categories and the "Yes: Hunt outside of breeding unit."However, we did not see this pattern in the data.The only part of this hypothesis that held true was that students with higher small population knowledge were 3 times more likely to fall into the "Yes: Hunt only outside of breeding area" category in all three assessment settings.The other decision categories were in the opposite pattern we expected.Again, we are unsure how to explain these data other than that students may not necessarily link their decisions to biological reasoning about populations.

/ 21
Other studies support that science content knowledge may not be a primary influence in environmental behavior and decision-making, but may serve a more indirect, intermediary role (Frick, Kaiser, & Wilson, 2004 ;Heimlich, Mony, & Yocco, 2013).Therefore, a student's content knowledge on a subject may not always solely predict decisions.When a controversial topic is relatively new, and students have limited background knowledge on the subject, it may be unrealistic to expect students to effectively integrate knowledge into their decision-making.We noticed that when students did explicitly incorporate ecological knowledge into their decision-making, responses were often not very sophisticated or reflective of an accurate understanding of mountain lion natural history or of the impacts different management strategies could have on mountain lion conservation.Students' discussion of scientific evidence relevant to apex predators, as well as an emphasis on considering genetic variability as part of the conservation decision, was infrequent and surface-level.

Limitations
We investigated several factors in relationship to the final decision, or stance, of the students at three time points in the class.Our intent was to gain some insight on the basis of students' decisions through this analysis.However, we recognize the complexity of students' reasoning and decision-making in relation to the simplicity of our analysis.Our analysis was correlative rather based on causation.The basis of students' decisions may be related to factors that we did not measure, for example, increases in knowledge in non-science domains that we did not measure such as policy, economics, wildlife management principles, etc.
One of our research goals was to try to understand if students were making decisions that aligned with identity variables, and made an assumption that alignment may signal the use of single-value heuristics, as opposed to a reasoned analysis using value trade-offs.However, our analysis did not directly measure students' process of making tradeoffs, and so there are limitations to the conclusions we can draw about this student practice.For example, there may have been male hunters from a rural area who performed a reasoned analysis of tradeoffs according to their values and still came to the conclusion that mountain lions should be hunted throughout Nebraska.And in that case, we would have not been able to detect a change in reasoning.However, our analysis does indicate that some small portion of students were less entrenched in predictable stances based on identity and value orientations, which overall may indicate that some new perspectives were taken on this issue and that some students may have had a new strategy for decision-making on this topic by the end of the course.Future research with alternative research approaches needs to be conducted to investigate students' practices of performing value tradeoffs, including qualitative analysis of student work or clinical interviews while decision-making.
Additionally, our scope of inference is limited to this particular unique course, population of students, and particular SSI context.It is likely that decision-making processes change, even among the same population with different SSI contexts depending on student motivation, interest, and perception of being a stakeholder.

CONCLUSIONS
In this study we sought to gain insight into students' decision-making practice and to further understand factors that relate to their ultimate decision about a complex SSI.We investigated this practice by determining what may predict undergraduate students' decision-making for a conservation-related socioscientific issue in unstructured contexts before and after the unit and in a structured context meant to support quality decision-making.As we hypothesized, and as previous studies in the decision-making sciences and science education literature supports, we found identity factors (gender, rural background, and hunting identity) to be variables that predicted student decisions in an unstructured decision-making setting before the course.Additionally, patterns for the relationship between decisions and gender, rural background, hunting identity, and value orientations generally remained the same throughout the course, which suggests these variables are strong predictors of stance.However, these identity factors were slightly less predictive at the end of the course as students overall moved to a more moderate position.There were not clear patterns of relationships between ecological knowledge and stance, which supports the notion that there is not a direct relationship between students' knowledge and attitudes, and that other considerations may be the primary lens through which students stances are formed.Overall these data do indicate that some students changed their thinking as a result of the course and may have increased consideration of tradeoffs leading to more informed, nuanced, and complex understandings of mountain lion conservation.
While this phenomenon needs to be further studied, it is an indication of a positive outcome of the course in that some of the students' decisions were no longer entrenched along dividing lines based on identity and social norms.Likewise, more research needs to be conducted to determine if instructional practices such the use of a SDM tool may support science literacy skill development.However, the significance of this work includes defining what encompasses high-quality decision-making (including the consideration of tradeoffs around multiple factors) for instruction as well as research.Additionally, this work provides methodological ideas about how to apply frameworks such as VBN theory to aid in detecting factors that predict students' decisions and to investigate the Alred & Dauer / Factors of undergraduate decision-making 14 / 21 impact of instructional tools to "move the needle" in terms of students' ability to think critically about important topics while managing their susceptibility to suboptimal decisions that are based on social norms or stakeholder in-groups.Extensions of this work call for teaching tools that facilitate students' ability to do accuracy-oriented reasoning that is open to scientific evidence, particularly when evidence may point to the need for solutions that run counter to students' personal viewpoints or identity.Likely these teaching tools are ones that enable to classroom environment to be one that fulfill students' basic psychological needs and emotions (Darner, 2019), and recognizes a clear role for students' values and position these values as relatable.This study begins to develops theoretical ideas about the role a science classroom could play in helping students examine and apply their own values to SSIs, begin to recognize the role cognitive biases play in suboptimal decisions, and gain the skill set to make higher quality decisions.
The complexity of SSIs warrants careful, informed decision-making from citizens.Because most day-to-day decisions require little intentional thought, sometimes citizens use these same impulsive decision strategies when making decisions about SSIs.To prepare students to become scientifically literate citizens who use deliberate strategies to make important decisions, we propose that the undergraduate classroom is an ideal environment where students can learn decision-making practices that can transfer to their future lives.The role of the specific decision-making tool we used, versus other classroom strategies, in supporting higher quality decisions will require more detailed or controlled analyses.While there is still much to learn about the implementation and potential benefits of SDM in the classroom, our research supports this practice as a worthwhile endeavor.

Pretest and Posttest Mountain Lion Management Opinion Prompt
Should we hunt mountain lions in Nebraska?Mountain lions have recently recolonized the Pine Ridge area in the northwest corner of Nebraska.Young male mountain lions have been documented throughout Nebraska including agricultural areas where suitable habitat may be limited.Nebraska Game and Parks recently opened a mountain lion hunting season in the Pine Ridge Unit in habitat that is suitable for mountain lions and where the population is growing.Last year there was a big debate in the Nebraska legislature around hunting mountain lions including issues of animal rights, human rights, safety, biodiversity and conservation.What do you think should be done about this problem?Should we hunt mountain lions in Nebraska?Why should we do it/not do it?0-student does not describes criteria, or offers criteria that are unrelated to the issue.2-student describes only one criterion, or the connection to the issue is unclear or not compelling, or the criteria are not wide ranging (missing an important aspect of economics, environment, ethics, society or other).4-student describes criteria with clear and compelling connections to the issue and uses criteria that demonstrate a wide-ranging view of the issue (examining important aspects of economics, environment, ethics, society or other) 3. Information (4 points) -Clarify the information known about possible alternatives, with particular reference to the criteria identified and to any scientific knowledge or evidence.
A) Do you have enough information about each option?What extra information, that wasn't covered in class, do you need to help you make the decision?B) Why is this information important to the issue?0-student does not offer a question that is important to the issue, or the question is not apparently related to the issue.2-student offers a question that is important to the issue, but does not explain adequately and accurately why it is relevant to deciding the issue.4-student at least one question that is important to the issue and adequately and accurately explains why it is relevant to deciding the issue.
4. Advantages/disadvantages (4 points) -Evaluate the advantages and disadvantages of each alternative against the criteria identified.Be sure to clearly describe the advantages and disadvantages of all of the options you lay out in #1.You may use the example table as a way to organize your response and modify it to fit the number of options and criteria you have (optional): Option #1 Option #2 Option #3 Criteria #1 Criteria #2 Criteria #3 0-student does not discuss advantages and disadvantages of all of the options they have laid out in #1, or the discussion of the tradeoffs are extremely thin.
2-student discusses advantages and disadvantages of most of the options, but is missing significant tradeoffs in terms of at least one criteria.4-student thoroughly discusses both the advantages and disadvantages of each option, and thoroughly addresses each criteria 5. Choice (2 points) -A) Choose an alternative based on the analysis undertaken.B) Why do you think this is the best option?
EURASIA J Math Sci and Tech Ed 21 / 21 0-the student does not provide reasoning for their choice, or the reasoning is weak, unclear and disconnected with the criteria and tradeoffs discussed above 1-the student provides reasoning for their choice that has some weak or unclear connections with the criteria and tradeoffs discussed above 2-the student provides clear and comprehensive reasoning for their choice that clearly links the choice with the criteria and tradeoffs discussed above Rank the issue on a scale of 1 (not at all important) to 10 (one of the most important issues)._______ 1-a rank given 9. (1 point) How important do you think this issue is to society?
Rank the issue on a scale of 1 (not at all important) to 10 (one of the most important issues).________ 1-a rank given http://www.ejmste.com

Figure 1 .
Figure 1.Change in decisions about mountain lion management (n=110).Circles indicate the number of students in each management decision.Arrow size indicates the number of people who changed decision category between the pretest and unit assessment and between the unit assessment and the posttest do YOU think?Should we hunt mountain lions in Nebraska?In this class you've read articles about this issue and you have had group and class discussions about what should be done about hunting mountain lions in Nebraska.Now take some time to use the "Slow thinking framework: steps for high quality decision-making" to outline your thoughts about the issue.Your thoughts below should be more clear and thought-out than what you did for group work in class, and will be graded more rigorously.What you write below should represent your own thinking, which may vary from the thinking of your group.1.Options (4 points) -List or identify the possible alternative courses of action in considering the problem or issue.Identify at least 3 distinctly different options.0-student does not describe more than one distinctly different option, or the options are outlandish and non-viable 1-student does not describe more than two distinctly different options, or one of the options is extremely unlikely or does not address a functional solution to one of the problems associated with the issue 2-student describes three possible and distinctly different options, each offer a functional solution to one problem associated with the issue 2. Criteria (4 points) -Develop or identify suitable criteria for comparing these alternative courses of action.The nature of these criteria is left open to discussion, but includes values.(To help you think about possible criteria, ask yourself: how are you going to choose between these options?What are the important things to consider?)Identify at least two criteria.

6.
Review (2 points) -Evaluate the decision-making process undertaken.A) What do you think of the decision you have made?How could you improve the way you made the decision?B) How does your decision solve a problem associated with the hunting mountain lion issue?C) Do you think your decision is viable?Why or why not? 0-the student offers no reflection or what is offered demonstrates no thoughtfulness 1-the student offers some reflection that demonstrates an understanding of the issue 2-the student offers reflection that demonstrates an understanding of the issue and thoughtfulness 7. (2 points) Is there anything you could do to impact this issue?What are some things you could do and how might they impact the issue?0-Noanswer.1-the action presented by the student are not clearly related to the issue.2-the action presented by the student is clearly related to the issue.8.(1 point) How important do you think this issue is to you personally?

Table 1 .
Summary of student value orientations

Table 3 .
Summary of student small populations knowledge sophistication

Table 4 .
Summary of variables predicting management decisions.Percent and mean scores (value orientations) of students are presented for each variable analyzed for significance in predicting management decision.Significant variables for pretest, SDM assessment, and posttest are in bold (multinomial logistic regression, p<0.1)