Design and validation of an instrument to determine the relationship between pedagogical content knowledge and practical work in science instruction

Research in science education has recognized the importance of pedagogical content knowledge (PCK) and its relationship to practical work. Although efforts have been made to characterize this relationship accurately, research in this field still needs to be completed. This article shows an instrument ’ s design and validation process that aims to determine the perception of PCK elements and their incidence in the design, implementation, and evaluation of practical work in science teachers. The questionnaire was subjected to content validation by a panel of experts and a pilot study to evaluate its internal consistency through Cronbach ’ s alpha. The construction and validation yielded an instrument of 30 items grouped into five categories: orientations toward science teaching, knowledge of the curriculum, knowledge of assessment, knowledge of students ’ understanding of science, and knowledge of instructional strategies. Reliability yielded a Cronbach ’ s alpha value of 0.881, showing the development of a consistent and reliable instrument.


INTRODUCTION
Pedagogical content knowledge (PCK) is a widely researched topic of interest in science education research.PCK is the knowledge that a teacher should have and that, together with the instructional practices associated with that knowledge, is intended to promote student learning of a particular concept of a science subject.Such knowledge is related to organizing, representing, and adapting a topic, problem, and question based on students' interests, misconceptions, and abilities (Bond-Robinson, 2005;Magnusson et al., 1999).Grossman (1990), Magnusson et al. (1999), and Shulman (1986Shulman ( , 1987) ) proposed a framework of PCK for science teachers that defined five categories: orientations to science teaching (OST), knowledge of students' understanding of science (SK), knowledge of science curriculum (CuK), knowledge of instructional strategies (ISK), and knowledge of assessment (AK).Since then, educational research aligned with this proposal has been responsible for conceptualizing, developing, explaining, and analyzing PCK in science teachers (Abell, 2007(Abell, , 2008;;Akinyemi & Mavhunga, 2021;Alvarado et al., 2015;Goes et al., 2020;Park & Chen, 2012;Park & Oliver, 2008;Sintema & Marban, 2021).Part of these developments has focused on identifying the role and relationship that practice has with PCK.In this sense, when PCK is used in teaching science practical work, a learning environment is generated that guides the teacher's actions in the teaching of a specific content or topic (Bond-Robinson, 2005).
Another aspect that stands out in practical work is its intentionality in the classroom (Abrahams & Reiss, 2012;Tacoshi & Fernandez, 2014).That is, if the work is focused on the learning goals or the goals set by the teacher.Thus, in line with what the students have to do, as opposed to what they do and what they learn (Abrahams & Millar, 2008).One of the objectives of practical work commonly cited in the literature (Hofstein & Lunetta, 2004;Lunetta et al., 2007) is to lead students to develop skills in handling laboratory instruments, to help them to awaken interest in science, to acquire scientific knowledge, to develop critical thinking and to promote scientific attitudes and values (Abrahams, 2017).
In this sense, practical work is an activity that a person performs to understand the phenomena of nature in which systematic forms of reasoning are learned, which are transferred to problem situations and allow the student to emulate the scientific role of research.Related to the above, in the context of practical work, it is essential to highlight the role of the teacher and his/her effectiveness.These roles (teacher and his/her effectiveness) have been addressed in educational research processes (Abrahams & Millar, 2008;Abrahams & Reiss, 2012;Abrahams & Saglam, 2010).For example, effectiveness has been studied from students' perceptions and the point of view of the results obtained from the understanding of teaching strategies (Baladoh et al., 2017).Thus, teachers must be able to use the knowledge of the students and their understanding about the purposes of teaching science.In such a way, these generated strategies are effective, focused on learning (Tacoshi & Fernandez, 2014).Moreover, the purpose of teaching with practical work in science is to develop thinking skills in students, starting from the sources of scientific knowledge (Gericke et al., 2022;Wei & Liu, 2018).Regarding the teaching role, research has focused on the perception of the teacher's conception of practice (Alneyadi, 2019;Shana & Abulibdeh, 2020) and on the perspective of PCK (Grossman, 1990;Shulman, 1986;Wei & Liu, 2018).PCK is characteristic of the teacher as an educational professional (Shulman, 1986(Shulman, , 1987;;van Driel et al., 2014).This knowledge of the teacher is the result of the experience that the teacher acquires over the years and the repertoires that he/she builds in the exercise of the profession (Abell, 2007;Grossman, 1990;Loughran et al., 2004).In identifying the elements of PCK, it is noteworthy that most of the reported works have been done qualitatively through classroom observations, documentary analysis, interviews, and case studies.In order to record and analyze the information from the context of the classroom and the teacher's perception, Loughran et al. (2004) proposed two instruments that allowed to record of the PCK in the exercise of pedagogical practice, which consist of guiding questions that consider the central ideas about teaching.These instruments collect the teaching objectives stated by the teacher, the students' conceptions and learning difficulties, the ways of approaching concepts, problems, and projects used by the teacher, and developing evaluation processes.The first instrument is called "content repertoires" (CoRe).The second instrument focuses on how the teacher materializes the CoRe and complements the answers to the guiding questions.It is called "repertoires of professional and pedagogical experience" (Pa-Pers) and consists of the answers to the CoRe instrument integrated with the evidence collected through interviews and classroom observations.These instruments are qualitative and allow to PCK elements to be captured without measuring them (Rollnick & Mavhunga, 2014).For example, in a case study, Wei & Liu (2018) recorded and analyzed PCK of a chemistry teacher in a hands-on classroom, where the participant had multidimensional and mixed orientations to teaching.The orientations shaped his knowledge of his students' learning, and the context significantly influenced his PCK.On the other hand, instruments have been designed, validated and implemented in which science teachers' perceptions have been characterized, firstly on the sources of PCK in teaching with practical work (Chen & Chen, 2021), and secondly on teachers' practical knowledge of practical work (Chen et al., 2022).Despite these reports of evidence of PCK elements in practical work, it should be noted that these reports have focused on a specific area (China).Moreover, it makes few studies about PCK elements and their incidence in  (Wei & Liu, 2018), to argue as "necessary" the design of instruments that can account for these aspects.
Therefore, this article developed a questionnaire to identify the perception of PCK elements and their incidence in the design, implementation, and evaluation of practical work in science teachers.

Initial Design of the Questionnaire
This study aimed to design and validated a questionnaire to identify the perceptions of science teachers about the components of PCK and its relationship with the design, implementation, and evaluation of practical work in science.
Two phases were implemented for the questionnaire design: Phase I was determined by selecting information in which the review and analysis of inputs previously used by another research were carried out.For this purpose, the contributions made by Chen and Chen (2021), Chen et al. (2022), Fan (2014), and Wei and Liu (2018) were considered.However, there was only a partial answer to the perception of PCK elements and their incidence in the design, implementation, and evaluation of practical work in science; therefore, the decision was made to design a new instrument.
Phase II was associated with the correlational aspect since the study variables were not intentionally manipulated.The purpose was to validate the questionnaire.
The questionnaire considers the components of PCK, which according to Magnusson et al. (1999), include: OST, CuK, AK, SK, and ISK.
The initial questionnaire consisted of thirty items.Each component consisted of six items and five variables.Each item has a 5-point Likert scale made up, as follows: 5: always, 4: almost always, 3: sometimes, 2: rarely, and 1: never.

Validation Procedure
The instrument was validated in order to verify its usefulness.For this purpose, the process of validation by expert judgment was used (Escobar-Pérez & Cuervo-Martínez, 2008).This subjection to expert judgment is characterized by the review of a specific number of experts who evaluate the items of the questionnaire and assess the level of adequacy of the items and the dimensions to be evaluated (Creswell & Creswell, 2018;Lester et al., 2014).
Each of the experts was provided with a matrix, which mentioned the intentions of the questionnaire, a brief explanation of the theoretical background, and the necessary instructions to carry out the validation.In addition, this matrix had a space for the experts to make observations and open comments on each item.The following aspects were considered: sufficiency, clarity, coherence, and relevance.Sufficiency refers that the items of the same dimension are sufficient to obtain the measurement of this dimension (Escobar-Pérez & Cuervo-Martínez, 2008).Each criterion was rated on a scale of 1 to 4, with "1" indicating that the criterion was not met; "2", low level; "3", moderate level; and "4", high level.The information obtained by each of the pairs was consolidated, the results were tabulated, and the consistency of the judges was analyzed.After collecting and consolidating the information from the judges, we proceeded, as follows: 1. Analysis of consistency among judges: for this analysis, frequency statistics (median and standard deviation) were determined to analyze the homogeneity obtained in the evaluations issued by the peer evaluators.
2. Item adequacy analysis: to assess the adequacy of the items to the content validity criteria, the "Aiken's V" coefficient (Aiken, 1980(Aiken, , 1985;;Penfield & Giacobbi, 2004), which measures the relevance of each item, was determined.It considers the number of dimensions evaluated by the experts and the number of participating judges, giving an overall assessment of the designed questionnaire.This statistic helps make decisions regarding modifying or eliminating items from a questionnaire.The algebraic modification proposed by Penfield and Giacobbi (2004) was used to calculate it: where X is the evaluation average between judges on each item; l is the lowest possible value (in this case, 1); and k is the range of possible values on the scale used (4-1=3).
The value obtained ranges from 0 to 1, where 0 indicates no expert agreement and 1 indicates perfect agreement among the evaluators (Benarroch et al., 2021;Escurra Mayaute, 1988).
The criteria for determining whether an item is eliminated or changed were, as follows: First, if the mean value is less than 3.2; second, if Aiken's V value is less than .75(Aiken, 1985); and finally, the qualitative observations of the experts merit the need to change or delete an item.

Validation Participants
The choice of experts is an appropriate step to establish content validity (Escobar-Pérez & Cuervo-Martínez, 2008;Pedrosa et al., 2014).The experts have a training profile associated with the highest academic degree (PhD in education, science didactics, innovation, and research in didactics and science chemistry) and have an essential trajectory in the field of research in science education.10 experts participated in this process.

Construct Validity
For construct validity, Kaiser-Meyer-Olkin (KMO) sample adequacy test was developed to verify the feasibility of the factor analysis.It was necessary to evaluate the relationship patterns of the items and to evaluate the behavioral patterns of the items.The correlation matrix allows us to observe the determinant of the matrix.When the value is close to zero, performing an association between items is not recommended.KMO index takes values between 0 and 1, and as a guide, the interpretation of the values are, as follows: <.5 unacceptable; between .5 and .6 poor; between .6 and .7 fair; between .7 and .8good and >.8 excellent (Guisande et al., 2013).According to the value it takes, it is possible to determine whether it is possible to resort to factor analysis as a strategy in the scale validation process.Also, Barlett's sphericity test was performed, making it possible to analyze whether the variables are correlated.

Psychometric Properties of the Questionnaire
The level of reliability of the instrument was determined.Subsequently, the internal consistency analysis was performed, which refers to giving significance to each of the items, and for this purpose, Cronbach's alpha value was determined.Microsoft Excel ® version 2019 and SPSS ® version 29 were used to analyze the results.

Pilot Participants
A sample of 110 students and graduates of undergraduate programs in chemistry, master's degree in chemistry teaching, and master's degree in teaching of exact and natural sciences from three universities located in the city of Bogotá and from a university located in the city of Talca (Chile) were used to carry out this analysis.The population sample was primarily female (59%).The average age was 31.63±11.09.About 64% are chemistry graduates, and about 22% are graduates in the training process doing pedagogical practice.At least 4% have professional training other than education.Almost a quarter of the sample has postgraduate disciplinary training, 21% have postgraduate training in education and pedagogy, and 31% are considering pursuing a postgraduate degree in the short term.The most frequent range of teaching experience was between one and five years (55%).The instrument was provided to each participant through a Google Forms ® link.
Each item was analyzed individually based on the sufficiency, clarity, coherence, and relevance findings and the qualitative evaluations generated by the experts.As an example, Table 2 shows the modifications for item SK1.This item obtained a value lower than .75 in clarity, sufficiency, coherence, and relevance.
Based on the results obtained from quantitative and qualitative evaluation of the pairs for each of the items, the test items that were modified were OST3, OST4, OST6, AK2, AK6, SK1, SK4, SK5, IK3, IK4, and IK5.

Construct Validation Analysis
Table 3 shows the results obtained from KMO test of sampling adequacy and Barlett's test of sphericity, which supports the factorial matrix.
The value obtained was .819,indicating a good correlation between the variables and allowing the factor analysis to be projected with the sample data.On the other hand, the instrument's significance with Barlett's sphericity value was less than .001,which confirms that the variables are sufficiently correlated due to the small significance values.
The statistical analysis associated with the variance allows us to confirm the existence of a data set, which is grouped into nine factors.These factors explain more than 65% of the total variance.It is also identified that nine eigenvalues exceed unity.The values obtained coincide with the subcategories of the PCK components.Table 4 shows the analysis result associated with the variance between variables.
Based on the results obtained, the final version of the questionnaire is shown in Table 5.

Reliability
The reliability of the instrument was assessed.For this purpose, an analysis of its internal consistency was carried out.Cronbach's alpha coefficient was determined and, based on the results obtained, concurrent criterion validity was performed through the total correlation of all items.The internal consistency assessment of the instrument yielded a Cronbach's alpha value of .881for the entire pilot questionnaire, indicating high internal consistency (Taber, 2018).The results are presented in Table 6.
In obtaining the final version of the questionnaire, the decision was made to review the items for modification or elimination.Those that showed an increase in Cronbach's alpha values and had a low homogeneity index (r<.30)were reviewed.Seven items cause a slight increase in the alpha value if deleted.

ISK2
The submission of practical work is done on an individual basis.

ISK3
Student is asked to submit a pre-report of practical work & is given feedback.ISK4 Awareness-raising activities are conducted prior to practical work.

ISK5
The student is familiarized with scientific article production.

ISK6
Practical work led to development of post-implementation feedback activities.
IK3, and IK5.However, because the elimination of the items would imply the loss of valuable information for the research process and a slight increase in the instrument's reliability, it was decided to keep all the questionnaire items.
Therefore, the 30 items described in Table 7 for the final version of the questionnaire are maintained and organized according to the categories described in Magnusson et al. (1999).

DISCUSSION AND CONCLUSIONS
This article illustrates the validation process of a questionnaire related to the perception of high school science teachers of the elements of PCK and its incidence in the design, implementation, and evaluation of practical work in science.
Considering the obtained results, the thirty items were organized in nine components that refers to PCK categories and subcategories.In the generation of the items, PCK categories and subcategories suggested in Magnusson et al. (1999) were taken into account: OST.This topic is related to the purposes that are set out in the teaching of a topic or content and the purposes of designing and implementing practical work in science are defined (Hofstein & Lunetta, 2004); knowledge of the curriculum refers to the goals and components that are mandatory.There, the teacher outlines a learning subject, as well as knowledge of the curriculum, i.e., what students should know and the relationship that is established from the goals for the design and implementation of practical work.Knowledge of evaluation, which are the ways that the teacher uses to evaluate his students, covering the aspects that are important to evaluate, as well as the selection of the contents that are evaluated and the methodologies that are used for this purpose.Furthermore, the criteria that are proposed for the evaluation of performance in science with practical work (Tacoshi & Fernandez, 2014).
SK covers what the teacher must know to promote the development of knowledge in students, the prerequisites and learning difficulties that arise when designing and implementing practical work in the teaching and learning processes.And finally, ISK, which addresses the general schemes by subject and specific ones by theme, which are the strategies to approach the design and implementation of practical work, which are helpful to lead students to the understanding of specific concepts.
In addition, the instrument was reviewed by ten experts to evaluate its sufficiency, clarity, coherence, and relevance for content validation.The respective modifications were made based on the evaluations obtained for each item and the judges' comments.In this sense, it is interesting to note that there was high agreement among the judges in the evaluation of each of the items, which shows that the proposed items are clear, sufficient, coherent, and pertinent to evaluate the perception of teachers concerning the design, implementation, and evaluation of practical work in science.
The result obtained for Cronbach's alpha was .881,suggesting a high value of internal consistency.Furthermore, it indicates that the questionnaire can help to measure the perceptions of PCK elements and their incidence in the design, implementation, and evaluation of practical work in science teachers.This consistency value is comparable to the obtained in other studies conducted for the measurement of PCK elements in science teachers (Chen & Chen, 2021;Chen et al., 2022;Irmak & Yilmaz Tuzun, 2019).The aforementioned comments have been presented an original validated instrument that allows identifying the perceptions of science teachers about the components of PCK and its relationship with the design, implementation and evaluation of practical work in science.Additionally, it allows teachers to have a process of reflection of pedagogical practices in the development of their teaching and learning processes with practical work, so that they can rethink and generate inputs that promote learning in students, as well as the development of scientific skills and attitudes (Alneyadi, 2019;Boyle, 2019;Hofstein & Lunetta, 2004;Tacoshi & Fernandez, 2014).
This instrument can be used as an input for the design and implementation of teacher professional development programs related to the design, implementation, and evaluation of practical assignments to gather teachers' perceptions and generate reflections on pedagogical practices, that can be applied to science teachers (chemistry, physics, and biology) who develop their teaching processes with practical work.

Contribution to the literature •
This study identifies science teachers' perceptions around the elements of PCK in practical work.•Itprovides a starting point in the development of teacher training programs, which generate processes of reflection of the elements of PCK in the design, implementation, and evaluation of practical work.•It provides a validated instrument that can be used in the professional development of teachers with PCK for chemistry, biology, and physics teachers who design, implement, and evaluate practical work.

Table 1 .
Expert evaluation results

Table 2 .
Expert evaluation & proposed changes to SK1 item "Initial item: Practical work promotes the use of materials from the student's daily life."This is more of a procedural objective than a conceptual one, which is what the category measures.EFA: Expand on the idea of "materials of life ..." EGZ: Revise the wording.It corresponds to the plural in the verb "to promote."SFS: It is recommended to clarify what is stated in the category.MQG: Practical work promotes using materials and resources from the student's daily life, such as …" "Modifications: Item modifications were made on results obtained in quantitative evaluation concerning criteria."Item's final composition: Practical work promotes using materials & resources in the student's everyday environment.

Table 4 .
Analysis of total explained variance

Table 4 (
Continued).Analysis of total explained variance

Table 5 .
Final version of questionnaire

Table 6 .
Instrument reliability statistics