Exploring the Coherence of Student Reasoning when Responding to Questionnaires on Thermally Activated Phenomena

Many research results show that students often highlight “mixed-type” reasoning when tackling problematic situations and problems. This reasoning is based on the simultaneous use of common-sense and mere descriptions of facts, perceived as sufficient to build an “explanation” of observed or proposed situations and problems. This fact can be interpreted as a lack of coherence. In this paper, we study the coherence of responses that a sample of undergraduate chemical engineering student give when they are asked to face real-life situations, to create explanations, and to use models in different contexts. We administered open-ended questionnaires before and after a twenty-hour Inquiry-Based workshop related to phenomena activated by a thermal overcoming of a potential barrier. Based on the Physics Education Research literature on student understanding of relevant physics contents, the student responses are analysed by using researcher-generated categories of reasoning and their coherence is studied. Finally, we discuss some implications of the results to improve the development of students’


INTRODUCTION
A relevant aim of university science programs is the development of student explanatory skills. They are fundamental for a meaningful understanding of science as well as for the development of professional competences. However, the development of these skills has always been an arduous objective to be pursued at any level of education because of the difficulty students face in effectively relating everyday experiences to a scientific theory (Nottis et al., 2010;Streveler et al., 2008).
Research has shown the relevance of characterizing the reasoning students in their first year of academic studies (freshmen) deploy when asked to create or use representations and explanations (Leach et al., 2000), their coherence (Bao et al., 2006;Engel-Clough & Driver, 1986;Maloney et al., 1993), and context-dependence (Redfors, 2003;Redfors et al., 2001), showing that freshmen are often inconsistent in their reasoning, even in situations that an expert would consider equivalent.
This and the awareness of the increasing role that research assigns to the processes of developing and using explanatory models in science education at all schooling levels led us to analyse, in a previous piece of research (Fazio et al., 2013), the reasoning deployed by engineering freshmen when asked to create representations and explanations for everyday life situations and phenomena.
The results showed that student reasoning could be classified into a few categories. Many students highlighted reasonings mainly based on commonsense/everyday-type reasonings or merely focused on the description of facts and formulas previously learned and memorized. A relevant number of students perceived the description of facts as sufficient to build an "explanation" of observed or proposed situations and problems. They clearly showed to have more than one view about nature and use of explications in science, often implementing reasoning strategies that are 2 / 11 inefficient at correctly connecting modelling to real situations, to build explanations. This behaviour has been described in the literature (e.g., Bao & Redish, 2006;Carley & Palmquist, 1992;Corpuz & Rebello, 2011;Hrepic et al., 2005;Maloney & Siegler, 1993), and highlights that students need to clarify differences and similarities between descriptive and explicative reasoning procedures to achieve authentic learning.
Moreover, other pieces of research have identified a general lack of coherence in the explanations and reasoning deployed by students to tackle problematic situations. This result is a relevant issue in science education, as argued by Phillips et al., which noted that coherence in explanations is a crucial component of scientific practice, as it is being able to identify gaps or inconsistencies in one's reasoning (Phillips et al., 2017).

RESEARCH PROBLEM
Based on the considerations described above, this study aims at elaborating on the coherence of the reasoning and explanations engineering freshmen deploy when they tackle the questions of the pre-and post-instruction questionnaires. Here, we mean that a student shows the highest coherence in tackling the questionnaire when he/she deploys a unique category of reasoning in responding to the different questions.

Research Hypothesis
The reasoning freshmen deploy when asked to build explanations to tackle a proposed situation/problem is often incoherent, even in situations that an expert would consider equivalent.

Research Question
To what extent can a teaching workshop based on implementing inquiry-based methods and sharing results with peers be useful in enhancing freshmen coherence in creating or using descriptions and explanations?

THE WORKSHOP
The workshop focused on physics and chemistry systems characterised by having two different states separated by an energy difference ∆E. A significant expression that describes these systems, quantifying the transition probabilities, is the Boltzmann Factor (BF) (Feynman et al., 1963). The workshop content mainly involved the electric current in materials (conductors and semiconductors) and vacuum systems (thermionic tubes). Situations where the Boltzmann Factor can be used to describe electrical conduction were analysed by following the "5E" instructional model (Bybee, 2006), with two of the authors as the instructors. The workshop phases are detailed in Table 1.
The students had already studied some electric conduction models like the Drude's and the Sommerfeld's ones (Griffiths, 1988) in their electromagnetism course. During the Workshop, they performed voltamperometric experiments on ohmic conductors, like metals. They also studied the resistivity vs. temperature relationship in these materials. Different interactions between the charge carrier and the lattice were discussed to make sense of the linear dependence of resistivity on the temperature in metals.

CONTEXT AND METHODOLOGY
Our research sample is made of 36 first-year students attending the Undergraduate Program in Chemical Engineering at the University of Anonymous. The majority of them attended secondary schools where physics is usually taught by following a traditional, teacher-centred approach based on the mere transmission of general concepts. Even when laboratory activities support a lesson, teachers use a demonstrative approach, not involving the students adequately.

Contribution to the literature
• In this paper, we aim to study the coherence of the reasoning engineering freshmen deploy when they tackle questions about thermally activated phenomena in both pre-and post-instruction questionnaires. • We draw implications about what part of an inquiry-based workshop can be considered more or less effective in developing and bolstering student explicative skills. Description of the activities Hours Engagement Presentation of the activities to carry out. Introduction to conduction in ohmic conductors and non-ohmic behaviour, as in semiconductor devices.
2 Exploration Students searched for information and planned their activities in small groups, formulating questions they would answer during the experimental activities. They were introduced to the laboratory and encouraged to explore the measurement facilities and materials available to design their own experiences. Students chose to address the electrical conduction process in vacuum tubes, which is not complex to study and shows marked non-ohmic behaviour.
During the 1st semester of their Degree Program, the students attended general mathematics, physics, and inorganic chemistry courses and passed the exams. When selected to participate in our study, they were attending a 2nd-semester Physics course dealing with the fundamentals of electromagnetism.
We analysed the reasoning deployed by the students by using two specially designed questionnaires. In each questionnaire, the first two questions deal with everyday life situations and with the activities carried out during the experimental phases of the workshop. The students have to pay particular attention to the physical quantities measured during experimental phases and involved in mathematical formulas used to fit experimental data.
Question three in both questionnaires explicitly ask for clarification about the catalyst concept, which should be well present in the students' background. The last three questions are about mechanical model building and simulation activities carried out during the workshop. They aimed at investigating students' ideas of modelling and the ability to apply the same model in different contexts that would be considered analogous by an expert.
We administered the first questionnaire at the beginning of the Workshop, and the second one at the end, after one month from the beginning. We classified students' responses by using some phenomenographic (Marton, 1986) categories. The construction of the Description of the activities Hours Explanation Students carried out their research investigations based on the hypotheses and questions formulated during the explorative phase. They studied the anodic current vs. the filament temperature to collect information about the values of concentration of electrons emerging from the filament. Mathematical modelling procedures were discussed to find a law to describe the concentration vs. temperature trend, which was found to contain the general BF expression. Some students searched for suitable models to make sense of the experimental evidence and of the specific form of the proper function they found, in particular concerning the meaning of the quantity "energy" in the function exponential term. They found references in manuals and on the internet to Richardson's law (Richardson, 1921) in vacuum tubes, described by an expression analogous to the mathematical function best fitting their experimental data. It contains the BF, and the students found that the "energy" reported in Richardson's Law exponential term is called the "work function," which is conceptually identical to the activation energy. The instructor also suggested analysing the energy band model in semiconductors and the energy gap concept by comparing it with the activation energy and work function concepts discussed before. After a group discussion, students were encouraged to focus on the idea of a "two-level" system as the unifying concept behind all the situations. Some time was devoted analysing an agent-based computer model (Battaglia et al., 2009) related to the subject, built by using the NetLogo simulation environment, which can easily simulate the interactions between a large number of elements. Students discussed with the instructor a mechanical model of a twolevel system. They dealt with many balls free to move on two connected planes, placed at different heights. The model is implemented by using the NetLogo simulation environment. It was possible to study the equilibrium distribution of the balls at the two levels and discuss the factors that influence this distribution. Finally, students compared the simulation findings, the experimental results, and the models explaining them.
10 Elaboration Students searched for physical and chemical situations different from those discussed during the previous activities, whose experimental dependence on temperature gives evidence of similarity with electrical conduction in semiconductors and thermionic tubes. Finally, each group wrote a scientific report. 3 Evaluation Students presented the most significant findings obtained from their experimental work and held a class discussion aimed at comparing and contrasting the results obtained by different groups.
2 Table 2. Pre-instruction questionnaire 1) A puddle dries more slowly at 20°C than at 40°C. Assuming all other conditions (except temperature) equal in the two cases, explain the phenomenon, pointing out what are the quantities needed for the description of the phenomenon and for the construction of an interpretative model of the phenomenon itself. 2) In chemical kinetics, it is well known that the rate of a reaction, u, between two reactants follows the Arrhenius law: = − Describe each listed quantity, clarifying its physical meaning and the relations with the other quantities. 3) What do you think the role of a catalyst is in the development of a chemical reaction? 4) Can you give a microscopic interpretation of the Arrhenius law? 5) Can you think of other natural phenomena that can be explained by a similar model? 6) Which similarities can be identified in the previous phenomena? Is it possible to find a common physical quantity that characterizes all the systems you discussed in the previous questions?

/ 11
analytical categories was based on previous research on student reasoning skills (Sperandeo-Mineo et al., 2006) and a careful reading of the students' responses within a framework provided by domain-specific expertise.
Each researcher independently analysed the students' responses and grouped them into three analytical categories. Each category identifies the lines of reasoning students deploy when they are asked to explain physical phenomena. The three categories and supported reasoning can be summarized as follows: -Everyday-type: it classifies students' responses in which personal, common-sense experience and everyday situations are mainly recalled. Here, the explicative power perceived by students is in the search for analogies with equivalent, familiar situations (individual explicative power). -Descriptive-type: Students giving these responses use descriptive-type reasoning that involves representations having characteristics of scientific models (search for relevant physical variables, their relationships, …), but based on a simple description of facts or on a recall of memorized formulas. The structure of these representations is unable to supply causal relations between physics quantities involved in a functioning model (i.e., microscopic/macroscopic).
-Explicative-type: This category classifies responses where students use explicative-type reasoning. These, in some cases, involve the envisioning of qualitative or quantitative relationships among physical quantities based on a cause/effect relation. In other cases, they provide an explanatory hypothesis which can be analysed at a theoretical level.
A fourth category was added to the three previously described, to take into account the answers not given.
The researchers compared the two groups of responses. In some cases, they classified a student's response differently. The disagreements happened 19 times, with an agreement of about 91%. Therefore, good inter-rater reliability of the analysis is obtained. The researchers compared and discussed the differences between the two groups and chose a unique shared categorization of responses.

THE RESULTS
Figures 1a and 1b show the distribution of the whole set of student responses into the three main categories described above for the pre-instruction and postinstruction questionnaires, respectively. The percentages of answers are calculated by also considering the answers not given. The pre-instruction Table 3. Post-instruction questionnaire 1) In modern oil mills, olive oil flows inside metallic pipes. These pipes are often enclosed in bigger, coaxial tubes in which hot water flows. Explain the possible reason for this, pointing out what are the quantities needed for a description of the proposed situation and the construction of an explicative model. 2) In chemistry, it is well known from Eyring's absolute rate theory that the viscosity of fluid follows the following law:

=
Describe each listed quantity, clarifying its physical meaning and the relations with the other quantities. 3) In the petroleum industry, additives are often added to gas oil to work as catalysts. What do you think can the role of these additives be in the flowing of gas oil in a pipe? 4) Can you give a microscopic interpretation of the ( ) law seen in question 2)? 5) Can you think of other natural phenomena that can be explained by a similar model? 6) Which similarities can be identified in the previous phenomena? Is it possible to find a common physical quantity that characterizes all the systems you discussed in the previous questions? Figure 1. Bar chart of the student responses given to the pre-instruction questionnaire (a) and post-instruction questionnaire (b), distributed in the three categories described below.

/ 11
set contains 20% fewer answers than the post-instruction one. Figure 1 shows that many student responses to both pre-and post-instruction questionnaires can be classified as Descriptive-type. However, about 15% of the answers to the pre-instruction questionnaire are classified as Everyday-Type. Such type of answers is missing in the post-instruction questionnaire, where we classified the remaining answers as Explicative-Type. These increase from about 13% to 40% going from the pre-to postinstruction questionnaire. Finally, our data highlight a general decrease of not-answered questions going from the pre-instruction (about 25% of total) to the postinstruction tests (about 10%). We related the degree of coherence of our student sample with the number of reasoning categories simultaneously used. These can range from a minimum of a single category (maximum coherence, when the student always uses the same reasoning category in answering the different questions of the questionnaire) up to four reasoning categories simultaneously used (minimum coherence). Figures 2 and 3 show this classification according to the criterium above described for the pre-and postinstruction questionnaire, respectively.
The results reported in Figure 2 clearly show that most students (just over 70%) employ at least three categories of reasoning when answering the questionnaire. This fact highlights a low consistency in the deployment of reasonings when answering the questionnaire questions. However, the distribution of students' reasoning categories shown in Figure 3 is quite different from that shown in Figure 2. We note that after instruction most of the students (about 60%) use at most two reasoning categories when tackling the proposed questions. Furthermore, no student uses four reasoning categories. Finally, it is important to highlight that, within the group of students corresponding to two categories, there are very few students not answering to all the questions.
Figures 4 and 5 show the distribution of student answers into the three categories described above for each question of the pre-instruction and post-instruction questionnaires, respectively.
We note that, before instruction, the most answered questions are the first three, with a relevant number of responses to question 2 classified as Everyday-Type. A general predominance of Descriptive-Type responses is evident, especially to question 3, where about 80% of students seem to deploy descriptive-type reasoning to tackle the question related to the catalyst concept. Questions four to six are answered by much fewer students than the first three, with only about 30% of the responses given to question 6. Again, Descriptive-Type responses are predominant.
After the workshop, we observe an increase in answers given to questions 4, 5, and 6. We also observe a general increase in Explicative-Type responses to all questions, even though we can't help but notice a persistence of Descriptive-Type responses, especially to questions 3 to 6.

DISCUSSION
Our results allow us to confirm our research hypothesis. The students in our sample deployed mixed categories of reasoning when building explanations to respond to the proposed questions. This behaviour is particularly true for the pre-instruction questionnaire, where just over 70% of students employed at least three categories of reasoning when answering the questionnaire. Furthermore, we note that even after instruction only 19% of the students gave fully coherent answers, in the sense of using a unique category of reasoning in responding to the different questions. Figure 2. Percentages of students simultaneously using from one to four reasoning categories (including a category for not-given answers) in answering the pre-instruction questionnaire Figure 3. Percentages of students simultaneously using from one to four reasoning categories (including a category for not-given answers) in answering the post-instruction questionnaire Sikorski and Hammer (2017) define coherence in building explanations as the process of "trying to build meaningful, mutually consistent relationships between information". This coherence is a fundamental feature of students' learning, as it can allow them to coordinate different perspectives in ways that provide new insight or understanding (Geller et al., 2019;Knorr Cetina, 1999;Sikorski & Hammer, 2017). The coherence in answering increases markedly after instruction. However, about 42% of the students still use more than two reasoning categories.
The results shown in Figures 1 to 5 also allow us to answer our research question. Besides the disappearance of Everyday-Type responses and more than doubling of Explicative-Type responses, our analysis shows that after instruction, the students are more coherent in giving responses using a distinct reasoning style or preponderantly using one, in some case explication-based. This preponderance is evident in responses to questions 1 and 2, the ones dealing with situations related to everyday life, and with the activities carried out during the experimental phases of the workshop, as shown in Figure 5.
However, we must note that a considerable number of students still give Descriptive-Type answers after instruction. This mainly happens in the answers given to questions from number 3 to 6. In question 3, we may suppose that the catalyst notion is taken into account only by recalling memories of past studies, without a clear understanding of the effect of a catalyst on the rate of a reaction (Çalik et al., 2010;Supasorn & Promarak, 2015).
The responses to the remaining questions and the argumentation given to support them highlight that many students have still not clearly understood the difference between a description and an explanation  (e.g., Besson, 2010;Karam, et al., 2011;Reese, 1999) of a given phenomenon. Moreover, some of the students seem to be sure that the "true knowledge of a phenomenon" only involves the ability to give some quantitative and detailed description of it, particularly by the use of mathematical language (e.g., Târziu, 2018).
After the inquiry-based instruction, the number of students responding to all the questions in the questionnaire significantly increased. Figure 5 also shows that answers to questions 5 and 6 considerably increased after instruction. These questions are the ones aimed at understanding if the proposed instruction is effective in favouring in students a transfer, or generalization, of learning, i.e., the development of the ability to take concepts learned in one context and apply them to novel problems in different contexts. This ability is well recognized as a key component to developing science knowledge applicable in many contexts of everyday life. Many theoretical models and empirical pieces of evidence (e.g., Zhao, 2012) imply that researchbased teaching methods such as Inquiry Based Science Education may play a key role in favouring the transfer of learning and in promoting the development of highlevel reasoning skills. Moreover, the increase in answers given to question 5 and 6 after instruction is paired with an increase of answering coherence, in the sense of the presence of only descriptive and explicative reasoning.
Our results also allow us to draw implications about what part of the Workshop can be considered more or less effective in developing and bolstering student explicative skills. To do that, we must look at the workshop structure and organization. It is mainly aimed at describing, analysing, and explaining several phenomena by using one common framework characterised by experimental and modelling activities. The students were also asked to present, discuss, and justify their most significant findings and conclusions to the colleagues. This request may have had a positive effect on the development of explicative skills, as already reported in the literature (e.g., Kuhn & Pease, 2008). We note here that, during the "explanation" phase of the Workshop (10 hours in total), the students studied the two-level analogic model through a computer-based simulation. However, in their answers to the questions, almost all the students maintaining Descriptive-Type reasoning status did not mention it. They always took into account the data collection activities and the mathematical modelling ones (data fitting). Only half of these students clearly showed in their response comments to be aware that a full understanding of the proposed phenomena requires the ability to figure out a mechanism of functioning (e.g., De Regt & Dieks, 2005; Dieks, 2019), like the one supplied by the mechanical model in the simulation. A close analysis of the 1 Analogy instruction is credited to be a very effective tool for promoting student conceptual changes and help them to store the concepts learned in their long-term memories (Çalik et al., 2010). responses given by the students to the questionnaire questions, and a preliminary analysis of interviews taken with some representative students of the sample, show that such a mechanical model seems to have supported students in understanding conduction in vacuum tubes and evaporation. However, the model appears to have been substantially less effective in helping the students in figuring out chemical and fluidity reactions/phenomena, mainly because of the lack of a clear visualization of a "spatial surface" to be escaped by particles in these two last cases.
These results prompted us to reconsider all the activities in the workshop. They allowed us to formulate hypotheses about modifications of activities that could bring students toward a substantial improvement of their explicative skills, as listed below: • to discuss more extensively during all the workshop activities that a mathematical or verbal representation of an observed phenomenon is only a description of it. To answer a question like "why and how a phenomenon is the way it is?" students need to figure out a mechanism of functioning, that in some cases can be represented by an analogic mechanical model 1 as that supplied by the simulation presented; • to allow the students to spend more time with the simulation, improving it to enable them to get a better representation of experimental situations in which there is not an escape of particles from a welldefined spatial surface. More time should be given to students to actively "play" with the simulation, modifying relevant parameters, discussing the consequent effect on the results, and comparing them with the experimental results; • to deepen and extend the discussion on the simulated model by making inferences about the model parameters and the physical quantities involved in the different phenomena analysed. Particularly, the mechanism of energy transfer to particles should be deepened and explicitly related to the interactions between electrons and atoms in the metal. The activation energy concept, known from previous studies, should be better linked to the energy gap one discussed in the simulated model.

CONCLUSIONS
Our research shows that in the initial phases of the Workshop many of the freshmen attending the chemical engineering graduate program at the University of Anonymous demonstrated mixed and incoherent reasoning skills concerning the understanding of phenomena. They are initially perceived as different, while being all analysable in the common framework of 8 / 11 the BF. Many students clearly showed to deploy mixed and not-coherent reasoning strategies, mainly focusing on a common-sense description of what they observed, and an uncontextualized use of mathematical formulas. The results of the experimentation of an inquiry-based workshop, using tools aimed at stimulating experimental analysis, as well as modelling at a micro level, seem to highlight the pedagogical efficacy of such an approach in enhancing the students' coherence in consistently using clear and well-defined reasoning styles. An increase of the number of questions answered by the students after instruction was also observed.
However, an accurate deepening of the proposed activities seems necessary. Particularly, we highlight the need to allow the students to have more time to directly compare the proposed analogic mechanical model, its results, and its characteristic physical quantities with the experimental results and the physical quantities characterizing the different phenomena. where students use explicative-type reasoning. These involve the envisioning of qualitative and/or quantitative relationships among physical quantities based on a cause/effect relation or providing an explanatory hypothesis which can be analyzed at a theoretical level.
Q1. If the temperature increases, the molecular energy is higher. So, more water molecules can overcome the evaporation energy barrier at 40°C rather than at 20°C. Q2. E is the activation energy (a description of the concept and its role in the formula is given), t is the temperature, k is the Boltzmann constant. Q3. As explained by the collision theory, a catalyst lowers the energy barrier the molecules must overcome to allow the reaction development... Q4. We know that gas is composed of particles that can collide with each other and gain enough energy to overcome the energy barrier... Q5. In chemistry, all reactions follow the Arrhenius law. The collision theory explains that when the temperature is high, collisions between particles are more energetic... Q6. The common physical quantities are temperature and activation energy.
In all these phenomena, there is threshold energy that must be reached to activate them, and the phenomenon rate depends on temperature…