The Relation Between ICT and Science in PISA 2015 for Bulgarian and Finnish Students

The relationship between Information and Communication Technology (ICT) and science performance has been the focus of much recent research, especially due to the prevalence of ICT in our digital society. However, the exploration of this relationship has yielded mixed results. Thus, the current study aims to uncover the learning processes that are linked to students ’ science performance by investigating the effect of ICT variables on science for 15-year-old students in two countries with contrasting levels of technology implementation ( Bulgaria n = 5,928 and Finland n = 5,882). The study analyzed PISA 2015 data using structural equation modeling to assess the impact of ICT use, availability, and comfort on students ’ science scores, controlling for students ’ socio-economic status. In both countries, results revealed that (1) ICT use and availability were associated with lower science scores and (2) students who were more comfortable with ICT performed better in science. This study can inform practical implementations of ICT in classrooms that consider the differential effect of ICT and it can advance theoretical knowledge around technology, learning, and cultural context.


INTRODUCTION
Science literacy, defined as the reflective citizen's ability to interact with the issues and ideas of science (OECD, 2017b), is essential for evaluating claims made by others or for assessing evidence for those claims (van der Linden, Maibach, Cook, Leiserowitz, & Lewandowsky, 2017). In an era in which people are constantly bombarded with facts, opinions, and information, consumers of information need to be able to sort through the falsehoods to find the truth and avoid being taken advantage of by "flim-flam" (Randi, 1982). In the 21st-century digital landscape, Information and Communication Technology (ICT) becomes an important tool in the process of discerning facts from fiction. Importantly, science literacy is the focus of the Programme for International Student Assessment (PISA) 2015, which is why we have chosen PISA 2015 as the source of the data for this research. The PISA assessment is conducted by the Organisation for Economic Cooperation and Development (OECD, PISA, n.d.). Science literacy is assessed in terms of a student's content, procedural, and epistemic knowledge of the scientific process (OECD, 2017a). There are three competencies nested under scientific literacy that PISA assesses: (1) explain phenomena using science, (2) design and evaluate experiments, and (3) interpret and evaluate data and evidence. The world OECD average for science is a score of 500 (SD = 100; OECD, 2016).
As 15-year-olds around the world live in an increasingly digital environment, PISA has also been concerned with assessing the role of information and communication technology in students' lives. Information and Communication Technology (ICT) is defined as a form of technology that stresses the role of communication, being also concerned with information storage and sharing. For instance, PISA defined ICT as "the use of any equipment or software for processing or transmitting digital information that performs diverse general functions, whose options can be specified or programmed by its user" (Organization for Cooperation and Development;OECD, 2005). ICT information of students is also collected by PISA via the ICT Familiarity Questionnaire (OECD, 2014b). ICT includes devices, 2 / 15 applications, networks, and systems that enable connection to information and to others. What differentiates ICT from other forms of information technology is its focus on communication. In many parts of the world, ICT is already changing the educational landscape (Johnson et al., 2014). Both parents and teachers recognize this change and support the incorporation of ICT in school learning, as they believe it will improve learning and the students' chances to be employable (European Union, 2019a). The current employment environment reflects the need for technology-capable workers who will stay relevant in the digital 21 st century (Cussó-Calabuig, Farran, & Bosch-Capblanch, 2018;ICTC, 2016;Nordicity, 2017;Yeganehfar, Zarei, Isfandyari-Mogghadam, & Famil-Rouhani, 2018).
However, current ICT access is not consistent across countries (OECD, 2015(OECD, , 2017a. Although ICT has a tremendous transformational potential for learning, it is also accompanied by issues regarding individuals' equitable access to technology. Different cultures may also be associated with different economies and levels of technology inclusion. For instance, one in five European students attends a school with access to high-speed internet (European Union, 2019a). In this study, Finland and Bulgaria were chosen because of their different economies and implementations of technology according to the World Bank Group in Growing United: Upgrading Europe's Convergence Machine (Ridao-Cano & Bodewig, 2018). In this study, we focused on two European countries with opposing ICT profiles of the 46 countries that participated in the PISA 2015 ICT Questionnaire. Nordic countries such as Finland continue to have better access to technology and be "digital frontrunners", whereas others like Bulgaria remain as "digital challengers" (European Union, 2019a, 2019b, 2019cNovak, Purta, Marciniak, Ignatowicz, Rozenbaum, & Yearwood, 2018, p. 3). There seems to also be a link to academic achievement as digital frontrunner countries often have higher classroom achievement than digital challengers. Separating countries based on digitization, growth potential, and digital prosperity yields a group of digital challengers at the low end of the spectrum, and digital frontrunners at the high end of the spectrum. Finland is considered a digital frontrunner because of the country's high level of technology inclusion, the small disparity between high and low technology users, and overall wealth (Novak et al., 2018;Ridao-Cano & Bodewig, 2018). Conversely, Bulgaria is considered a digital challenger due to the large gap between high and low technology users, their late emergence into a free market, and their lower school achievement. The European Union reports (2019b, 2019c) reveal that the high-achieving country of Finland had higher student and teacher digital competency than both the European average and Bulgaria. Finland is much more digitally equipped than Bulgaria, but the two countries exhibit similar computer usage. This suggests that the effect of ICT may be deeper than its sheer usage (European Union, 2019b, 2019c. Several studies have found positive associations between certain ICT variables and science performance in PISA data since the first administration of this test in 2000. For instance, researchers found that interest in ICT as well as self-perceived ICT competence and autonomy were positively associated with science scores (Hu et al., 2018). It would seem that ICT in schools leads to better learning and better employability upon graduation. However, the relationship between academic scores and ICT is not so straightforward. For instance, a positive relationship between ICT use at home for schoolwork was only found for top-performing countries (Petko et al., 2017). Others found that ICT use for entertainment at home as well as the availability of ICT at home and at school were associated with science scores in some countries but not in others (Bulut & Cutumisu, 2018). Also, ICT does not always lead to better student learning (Cheung & Slavin, 2013;De Witte & Rogge, 2014;Hu et al., 2018).
Thus, it is important to determine which aspect of ICT has a larger effect on students' science performance. This will serve as a first step to identify the sources of some of the inconsistent results between ICT and science performance. Consequently, the study explores the relationship between ICT and science performance in a large international standardized achievement test for students from two widely-different countries in terms of digital proficiency. More specifically, this research explores whether factors such as ICT use, availability, interest, competence, autonomy, and social use are associated with higher or lower science scores in students from Bulgaria and Finland. The main research questions posed by this study are the following: To what extent does ICT help or hinder students' science performance? How does the relationship between ICT and science performance differ in the context of two

Contribution to the literature
• This study employs structural equation modeling (SEM) to investigate the contribution of ICT on students' science scores in Bulgaria and Finland in PISA 2015, controlling for the influence of socioeconomic status (SES). • In both countries, ICT use and availability were associated with lower science scores.
• In both countries, students who were more comfortable with ICT performed better in science.
3 / 15 countries with disparate access to ICT? We hypothesize different patterns of the relationships between ICT and science in Finland and Bulgaria, due to the differential availability, access, and familiarity with technology in the two countries.
This study makes several contributions to the literature. First, it employs structural equation modeling (SEM) to rigorously investigate the unique contribution of each of the ICT variables on students' science scores. Then, this study uses nine ICT variables measured by PISA's ICT Familiarity Questionnaire (OECD, 2014b) and it also accounts for the influence of socio-economic status (SES). Finally, the study explores both positive and negative relationships between ICT and science in these countries to provide a starting point to interventions that may enhance students' science performance.
The manuscript is organized as follows. First, it starts with a description of the theoretical framework underpinning this research. Second, it presents a review of the related literature on the relationships between ICT variables and science performance in PISA data. Third, it presents the methods employed to analyze the data, followed by the results. Finally, it concludes with a discussion of the results, their limitations, and directions for future work.

THEORETICAL FRAMEWORK
This study draws on several theories. The cultural context of students' use of technology included in this research is influenced by the cultural constructivist learning theory, where students take an active role in creating their knowledge that is culturally situated (Cobern, 1993). For instance, learners may use the technology available in their cultural context in different ways. According to Piagetian (1952) constructivist learning theory, students learn by creating knowledge from their environment through assimilation and accommodation. Therefore, in this research, students use the ICT available to them in their environment to create knowledge so they can succeed in school.
Self-determination theory (SDT) is a motivation theory suggesting that self-motivation is the causal factor behind an individual's success in learning (Ryan & Deci, 2000). In other words, an individual's determination will govern the amount of energy that is used to attain a positive outcome. This theory provides a useful lens for the examination of the ICT factors at the core of the present research. For an individual's comfort with ICT, PISA measures a student's perceived competence and autonomy, as well as interest and social interactions using ICT. Similarly, SDT involves three core psychological needs: competence, autonomy, and relatedness. High competence indicates that a student is more in control and has a high level of mastery when using ICT. High autonomy occurs when a student wants to make their own choices and be independent around ICT. High relatedness stems from students' use of ICT to connect to others, measured by PISA as the frequency of ICT incorporation into social interactions.
Social cognitive theory (SCT; Bandura, 1986) informs this research, as it attempts to explain how individuals regulate their behavior to achieve and maintain goaldirected behaviors over time. Thus, students' selfefficacy (i.e., confidence in one's ability to perform a behavior successfully) in using ICT for learning and entertainment could also play an important role in how they engage with ICT and how they set their goals. This theory is relevant to this study because it posits that individuals learn within a dynamic, social context through their interactions with their environment, being influenced by their past experiences.
Lastly, an ecological framework was used to emphasize the role of teachers and the importance of the learning environment in technology-rich settings, which has implications to how ICT is used in schools (Zandvliet, 2012). Its conceptual model consists of three spheres of influence, the ecosphere (i.e., an individual's physical environment that can influence the types of pedagogies employed), the sociosphere (i.e., an individual's interactions with others within that environment), and the technosphere (i.e., all humanmade artifacts in the world).

LITERATURE REVIEW
Previous research has been conducted to examine the relationship between ICT and school achievement, as part of a larger area of research that focuses on several factors that influence ICT. However, some of the results from different studies are contradictory and few are wholly in favor or opposed to ICT inclusion for students. Results are mixed regarding the positive or negative influence of ICT on students' science scores. Findings are presented by ICT variables measured by PISA. Petko, Cantieni, and Prasse (2017) analyzed all the 39 countries that participated in PISA 2012 using multiple linear regression with weights provided by PISA to investigate the links between ICT use, attitudes, and academic achievement. Hu, Gong, Lai, and Leung (2018) examined the relationship between students' academic achievement and ICT for 44 countries that participated in PISA 2015, using a three-level hierarchical linear model on the nested data (student, school, and country levels). Luu and Freeman (2011) use 2006 PISA data to compare ICT and science scores between Australia and Canada, using and exploratory factor analysis (EFA) and hierarchical linear model. Bulut and Cutumisu (2018) examined PISA 2012 results of ICT and academic achievement for Finland and Turkey using SEM in MPlus. Meng, Qui, and Boyd-Wilson (2018) looked at the relationship between ICT and academic achievement in Chinese and German students from PISA 2015. Finally, Zhang and Liu (2016) uncovered the effects of ICT on 4 / 15 academic achievement in the PISA data from 2000 to 2012 using three hierarchical linear models and controlling for several confounding variables. Gamazo, Martínez-Abad, Olmos-Migueláñez, and Rodríguez-Conde (2018) used multilevel modeling to analyze the PISA 2015 data for Spanish students.
The majority of the studies reviewed here use at least some of the nine variables examined in the current research that represent students' interactions with ICT measured by the ICT Familiarity Questionnaire from PISA 2015 (OECD, 2014b). The questionnaire began in 2003 and, over the PISA iterations, more specific questions were added to gain a bigger and clearer picture of how students interact with ICT (OECD, 2005(OECD, , 2009(OECD, , 2014a(OECD, , 2017b. The HOMESCH variable (variable IC010) denotes the use of ICT at home for schoolwork (OECD, 2014b(OECD, , 2017b. The variable asks the following: "How often do you use digital devices for the following activities outside of school?" An example of an item within this variable is IC010Q01TA: "Browsing the internet for schoolwork (e.g. for preparing an essay or presentation)." with possible answers of "Never or hardly ever," "Once or twice a month," "Once or twice a week," "Almost every day," or "Every day." The ENTUSE variable (IC008) denotes students' ICT use for entertainment at home (OECD, 2014b(OECD, , 2017b. The variable asks the following: "How often do you use digital devices for the following activities outside of school?" An example of an item within this variable is IC008Q05TA: "Participating in social networks (e.g. Facebook, Myspace)" with possible answers of "Never or hardly ever," "Once or twice a month," "Once or twice a week," "Almost every day," or "Every day." The USESCH variable (IC011) denotes students' ICT use at school (OECD, 2014b(OECD, , 2017b. The variable asks the following: "How often do you use digital devices for the following activities at school?" An example of an item within this variable is IC011Q02TA: "Using email at school." with possible answers of "Never or hardly ever," "Once or twice a month," "Once or twice a week," "Almost every day," or "Every day." In PISA 2015, the ICTHOME variable (IC001) denotes the availability of ICT devices in students' home (OECD, 2014b(OECD, , 2017b. The variable asks the following: "Are any of these devices available for you to use at home?" An example of an item within this variable is IC001Q01TA: "Portable laptop, or notebook" with possible answers of "Yes, and I use it," "Yes, but I don't use it," or "No".
The ICTSCH variable (IC009) denotes the availability of ICT devices in a student's school (OECD, 2014b(OECD, , 2017b. The variable asks the following: "Are any of these devices available for you to use at school?" An example of an item within this variable is IC009Q11TA: "Interactive whiteboard, e.g. SmartBoard" with possible answers of "Yes, and I use it," "Yes, but I don't use it," or "No".
The AUTICT variable (IC015) denotes students' selfreported autonomy around the use of ICT (OECD, 2014b(OECD, , 2017b. The variable asks the following: "Thinking about your experience with digital media and digital devices: to what extent do you disagree or agree with the following statements?" An example of an item within this variable is IC015Q02NA: "If I need new software, I install it by myself" with possible answers of "Strongly disagree," "Disagree," "Agree," or "Strongly agree".
The COMPICT variable (IC014) denotes students' self-reported competence in ICT tasks (OECD, 2014b(OECD, , 2017b. The variable asks the following: "Thinking about your experience with digital media and digital devices: to what extent do you disagree or agree with the following statements?" An example of an item within this variable is IC014Q03NA: "I feel comfortable using digital devices that I am less familiar with" with possible answers of "Strongly disagree," "Disagree," "Agree," or "Strongly agree".
The SOIAICT variable (IC016) denotes students' inclusion of ICT as a topic in social interactions (OECD, 2014b(OECD, , 2017b. The variable asks the following: "Thinking about your experience with digital media and digital devices: to what extent do you disagree or agree with the following statements?" An example of an item within this variable is IC016Q05NA: "I like to share information about digital devices with my friends" with possible answers of "Strongly disagree," "Disagree," "Agree," or "Strongly agree".
The INTICT variable (IC013) denotes students' interest in ICT (OECD, 2014b(OECD, , 2017b. The variable asks the following: "Thinking about your experience with digital media and digital devices: to what extent do you disagree or agree with the following statements?" An example of an item within this variable is IC013Q012NA: "I feel bad if no internet connection is possible" with possible answers of "Strongly disagree," "Disagree," "Agree," or "Strongly agree".

Positive Relationships
Petko et al. (2017) reported a positive relationship between ICT use at home for homework (HOMESCH) and science only for top-performing countries. Rodrigues and Biagi (2017) examined 25 European countries in PISA 2015 and found that European students with low levels of ICT use also tend to perform better in science. Hu et al. (2018) discovered that students who use ICT devices at home for entertainment purposes (ENTUSE) achieved higher science scores. Bulut and Cutumisu (2018) also found that ICT use for entertainment at home was positively correlated with science scores in Turkey.
Availability of ICT at home (ICTHOME) and at school (ICTSCH) were positively correlated with science scores in Turkey (Bulut & Cutumisu, 2018). ICT autonomy (AUTICT) was found to be a positive predictor for science scores at the OECD average level (Hu et al., 2018) as well as at the country-specific level in China and Germany (Meng et al., 2018). Self-perceived competence (COMPICT) also positively predicted overall OECD countries' science scores, as reported by Hu et al. (2018). Luu and Freeman (2011), using slightly different measures, found that an increase in competence towards aspects like basic ICT tasks and presentation software positively predicted science scores in Australia and Canada. Finally, a positive association between students' interest in ICT (INTICT) and their science scores was found at the overall OECD level (Hu et al., 2018) as well as at the specific country level for Chinese students (Meng et al., 2018).

Negative Relationships
Petko et al. (2017) reported a negative relationship between ICT use at home for homework (HOMESCH) and science for the majority of the low-performing countries. Hu et al. (2018) found overall lower science scores linked to ICT use at home for schoolwork. Luu and Freeman (2011) discussed ICT use at home for homework having a negative relationship with science scores for Australian students other than the general use of computers. Petko et al. (2017) found that ICT use for entertainment at home (ENTUSE) negatively predicted science scores using the OECD countries' average for PISA 2012. Bulut and Cutumisu (2018) also found that ICT use for entertainment at home was negatively correlated in Finland in PISA 2012. Both Petko et al. (2017) and Hu et al. (2018) found a negative relationship between a student's science scores and ICT use at school (USESCH). Bulut and Cutumisu (2018) also found a negative relationship between ICT use at school and science scores using country-specific data for Finland and Turkey. Luu and Freeman (2011) found that all forms of high ICT usage other than browsing the internet were associated with lower science achievement in Australian students. For students in 25 European countries, medium and high users of ICT performed worse on science assessments than low users of ICT in PISA 2015 (Rodrigues & Biagi, 2017).
At the OECD country level, Petko et al. (2017) found a negative relationship for ICT availability at home (ICTHOME) and science achievement. Meng et al. (2018) present a negative relationship for ICT availability at home and science for German students.
Finally, Meng et al. (2018) state a negative relationship between competence and science scores for Chinese students. Students' inclusion of ICT as a topic in social interactions (SOIAICT) was found to be a negative predictor of science scores for individual countries (Meng et al., 2018) as well as on a larger scale (Hu et al., 2018).

Null Relationships
A null (i.e., neutral) relationship for Canadian students was found by Luu and Freeman (2011) when examining ICT use at home for homework (HOMESCH) and science. Bulut and Cutumisu (2018) also found no link between science scores and ICT use at home for schoolwork. Luu and Freeman (2011) found that the relationship between ICT use at school (USESCH) and science scores was not significant in Canada. Bulut and Cutumisu (2018) found that the availability of ICT at home (ICTHOME) was not related to science scores for Finnish students. There was no significant relationship between science and ICT availability at school (ICTSCH) at the OECD country level (Hu et al., 2018) as well as in Finland (Bulut & Cutumisu, 2018).
Finally, Meng et al. (2018) found no relationship between competence and science scores for German students.
While the literature review we have conducted found that many countries have been the focus of ICT and STEM (Science, Technology, Engineering, and Mathematics) education research, there is little consensus as far as positive, negative, or null results for the nine separate ICT variables. In addition, few studies examined all ICT variables simultaneously in the context of science education. The most common method in the articles reviewed is hierarchical linear regression, which is less rigorous than the SEM used in the current research. AUTICT was the only variable where no negative or null relationships were found. COMPICT was the variable with the second highest report of positive relationships with science scores. According to the studies reviewed, the ICT use variables held mainly negative relationships with academic scores. Some inconsistencies in results may be due to the difference in a country's definition or implementation of ICT for their students. Thus, the wide variety of ICT relationships with science scores between countries with differing ICT profiles has not been thoroughly explained despite its occurrence in several reviewed articles.

Data Source & Sampling
This research employs data from Finland and Bulgaria, two countries that participated in the ICT Familiarity Questionnaire of the PISA 2015 database (OECD 2014b(OECD , 2017b. PISA is an assessment implemented by the OECD that collects data from 15year-old students in countries around the world every three years to evaluate students' competence levels in mathematics, science, and reading (OECD, 2017b). This test consists of both constructed-response items and 6 / 15 selected-response items. PISA is designed by experts in measurement, evaluation, and various content areas to test student performance and whether students are able to apply their knowledge (OECD, 2017b). The results are also used to promote equitable learning worldwide and inform decision making for educational policy. Specifically, the science variable used in this study is defined by OECD (2016) as follows: "A scientifically literate person is willing to engage in reasoned discourse about science and technology, which requires the competencies to explain phenomena scientifically, evaluate and design scientific enquiry, and interpret data and evidence scientifically" (p. 13).
The surveys are also used to measure other related variables to education, such as students' interactions with ICT, the focus of the present study. The age of the students ranges from 15 years and 3 months to 16 years and 2 months. The test items are delivered as computerbased questionnaires and quizzes. The data used for this research included the ICT Familiarity Questionnaire (OECD, 2014b) and the Student Questionnaire (OECD, 2014c). The sample consists of a total of n = 11,810 students (Bulgarian n = 5,928 and Finnish n = 5,882) who attended school part-time or full-time in their countries (OECD, 2016(OECD, , 2017b. Participants were randomly chosen within their cluster sample of schools. At least 150 schools were sampled per country. Sampling weights for students were also used to compare students from different schools to ensure that the data are correctly treated (e.g., all countries were equally represented) and that sampling error is properly calculated.

Measures
Science. The dependent variable, science score, reflects a student's ability to use scientific knowledge to explain phenomena scientifically, evaluate and design scientific inquiry, and interpret data and evidence scientifically (Gurria, 2016). Students were presented with translated questions from a pool of science-based items to test their knowledge. The scores of the students were analyzed using Item Response Theory (IRT) to obtain a scale on which students could be compared. For each student, PISA computed ten plausible values for science based on students' answers to the science items and other questions in the survey to represent their knowledge in each of the science competencies measured. These values were used as indicators of a latent science score dependent variable. As the scale proficiency values for science are not observed, PISA treated them as "missing" data (Rubin, 1987). Thus, PISA uses plausible values for science, calculated by multiple imputation from science questions and background information, aiming to estimate a student's science performance from individual item scores as well as to better represent the variability in students from schools, areas, and countries. More details on the necessity and use of plausible values are included in the PISA Technical Report (OECD, 2017b). As PISA statisticians have already produced the plausible values for science, we used each of the ten plausible values as latent variable indicators to get a more accurate estimate of what the latent value for Science would be.
The results from all schools are standardized so that schools and countries can be compared on a scale of 0 to 1000 with a global mean of 500 and standard deviation of 100. As per the guidelines of the PISA Technical Report (OECD, 2017b), plausible values will only be used to compare students at the population level, not at the individual level.
ICT. The observed independent variables are derived variables built by PISA (OECD, 2017b, p. 328). They are responses to nine Likert-scale questions (discussed previously) from the ICT Familiarity Questionnaire (OECD, 2014b) and are reported by PISA as logit scales, where zero represents the OECD average (OECD, 2017b). The PISA 2012 Technical Report presents a detailed description of these measures and the methods employed to create them (OECD, 2014a(OECD, , 2017b. The ICT Familiarity Questionnaire contains 15 items in total. However, six items (IC002-IC007) were not used in this study, as they were either not included in the articles in the literature review or they did not fall under the scope of the three components of SDT.

Economic, Social, and Cultural Status (ESCS).
This measurement is constructed by PISA using principal component analysis (PCA) from several other items throughout the survey (OECD, 2017b). The components are parental education (PARED), highest parental occupation (HISEI), and home possessions (HOMEPOS). PARED asks students if their parents have no education, primary education, lower secondary education, vocational/pre-vocational upper secondary, general upper secondary, non-tertiary post-secondary, vocational tertiary, or theoretically-oriented tertiary and post-graduate. HISEI asks the students an open ended question of both their parents' employment. The responses were interpreted and coded as a four digit code to enable cross cultural comparisons. The HOMEPOS variable was comprised of 16 questions that asked about the amount and types of items in each student's home. These three combined variables allow researchers to make an inference to family income or wealth. The ESCS scores are standardized for a standard deviation of one and so that a value of zero represents the overall OECD average. In Bulgaria, the mean ESCS is -.06 with a standard deviation of .98. In Finland, the mean ESCS is .26 with a standard deviation of .75. This measure will be used as an indicator of student SES in the current study.

Analysis
This study employed SEM to explore the relationship between ICT and science performance, controlling for students' socio-economic status. This method was used because of its robustness to non-normality in the data (Kline, 2015). This analytic method allows for a combination of CFA and path analysis and it is less laborious than other linear equation methods. Missing values in the ICT Familiarity Questionnaire (OECD, 2014b) data were replaced with a value of -999 in preparation of using the SEM software. This ensures that the program recognizes -999 as missing values in the dataset. Then, the data was loaded into the SEM program MPlus 7 (Muthén & Muthén, 2012) to model the proposed relationship. MPlus accounts for all standard procedures in weighting (Muthén & Muthén, 2012). Maximum likelihood estimation with robust standard errors (MLR) was used with the weighting variable provided by PISA to ensure that countries can be compared to each other. The missing data is assumed to be random and was treated with Full Information Maximum Likelihood (FIML), which determines likely parameter estimates based on the associations of all variables in the model. Phase 1. The purpose of this phase was exploratory, aiming to provide a jumping-off point to create a better explained model. At first, a confirmatory factor analysis (CFA) using the ten plausible values for science predicted by all nine observed ICT variables was conducted. Results indicated a good model fit but many of the predictor variables were highly correlated, warranting a factor analysis. Thus, Phase 2 was initiated to determine the distinct ICT dimensions among the observed ICT variables using exploratory factor analysis (EFA).
Phase 2. The aim of this phase is to identify the dimensions of the ICT variables, which constitute the latent variables included in the SEM analysis. Latent variables represent constructs that cannot be measured directly. They can be examined through the SEM analysis of their corresponding observed ICT variables and they help better explain the variance in the science scores. Thus, an EFA analysis involving only the nine ICT variables (without including the ten science plausible values) was conducted in MPlus to investigate whether any of the variables could be combined into latent factors. The EFA showed that the three use variables, the two availability variables, and the four comfort variables could be combined as three separate unobserved latent factors, respectively. The new latent predictor variables were named ICTUSE, ICTAVB, and ICTCOMF, respectively.
Cronbach's alpha (α) is calculated by OECD and recorded for each variable to compare internal consistencies among countries (OECD, 2017b). A value of 1 signifies perfect internal consistency, while a value of .7 indicates acceptable internal consistency. ICT use reflects how often types of ICT are used by students. This includes ICT use at home for school work or entertainment as well as ICT use at school. PISA records these variables as HOMESCH (α = .95 in Bulgaria and .92 in Finland), ENTUSE (α = .87 in Bulgaria and .80 in Finland), and USESCH (α = .93 in Bulgaria and .85 in Finland), respectively. ICT availability is a measure of the types of ICT that the students have access to at home or at school and whether or not they use it. In PISA, these variables are labeled ICTHOME and ICTSCH, respectively. ICT comfort denotes how autonomous and competent students believe they are, how often they use ICT as a topic in conversation, and how interested they are in ICT. PISA has labeled these variables as AUTICT Phase 3. This phase is necessary because the observed predictor ICT variables are measured in a similar fashion and cover similar aspects of ICT, therefore there could be some shared variance due to their similarity. SEM was used to control for the shared variation of the ICT observed variables and allow for only the unique contribution of these predictors. Thus, another CFA analysis was conducted using MPlus with the three ICT latent variables representing the nine observed ICT variables, predicting the science latent variable (composed of the ten science plausible values). The ShareICT is a latent variable that was created in this phase as a combination of all observed ICT predictor variables, allowing the model to partial out the common variance among the ICT measures and their unique contributions to be analyzed. By doing this, the model was turned into a bifactor SEM (Holzinger & Swineford, 1937;Kline, 2015) that seeks to control for the shared contribution of the observed variables (i.e., is a single source of common variance shared across the observed variables), while simultaneously considering each of the three unique ICT domains. Thus, this bifactor model, in addition to a unidimensional model, can test whether test-unit specific factors need to be considered (OECD, 2017b, p. 167). In sum, the CFA analysis in this phase enabled the creation of a bifactor model that will be used in Phase 4.
Phase 4. In this phase, data from both countries were combined into a multi-group bifactor model. Parameters were constrained in the model to achieve model convergence and evaluate measurement equivalence. The model showed strong measurement invariance and revealed that the same indicators can load onto the same factors in both countries, that the unstandardized factor loadings are invariant, and that the estimated indicator intercepts are invariant. These constraints did not significantly worsen model fit. An index of economic, social, and cultural status (ECSC) provided by PISA 2015 was also added as a control variable so that students' 8 / 15 economic status would not bias the interpretation of the association between the ICT variables and science scores (OECD, 2017b, p. 339). Figure 1 illustrates the diagram of the final model.

Assumptions.
Correlations of the nine predictor variables were examined for both countries. All correlations shown in Tables 1 and 2 were below the Kline's (2015) collinearity cutoff of .85, therefore collinearity was not a concern. The plausible variables for science were highly correlated, as they measured the science knowledge of the same student. Therefore, all ten variables were combined into an overall science latent variable, named Science. The significant correlations for predictor variables ranged from .03 to .63, as shown in Tables 1 and 2.  Skewness and kurtosis were measured and included in Table 3, along with data on missing values.
Four ICT measures in Finland were beyond the recommended ±2 standard deviations cutoff for normality, with only two in Bulgaria (George & Mallery, 2010;Little, 2013). Finland's high kurtosis will be discussed in the Limitations section.

RESULTS
The model fit statistics used in this study are the Comparative Fit Index (CFI) and the Root Mean Square Error of Approximation (RMSEA). The Chi-square test was not used to evaluate model fit because it is a very sensitive test and provides meaningless results when used with very large samples (Hayduk, 2014;Little, 2013). CFI compares the hypothesized model with an independent model with no specifications. For CFI, a value of 1 is a perfect fit, while values equal to or greater than .9 are a close fit. RMSEA estimates the overall lack of fit, with .06 or below being considered as good fit. Some of the predictor variables yielded correlations upwards of .63, which enabled the combination of variables into fewer factors. Phase 1. The initial CFA model fit the data well. The model consisted of the nine observed variables predicting a Science latent variable made up of the ten plausible values for science. The CFA was 1.00 for both countries, while the RMSEA was .008 for Finland and .007 in Bulgaria. However, this initial model only explained 1.66% of the variation in Bulgarian science scores and .54% in Finnish science scores.
Phase 3. A second CFA analysis was conducted which incorporated the three new latent predictors (ICTUSE, ICTAVB, and ICTCOMF) to produce a latent variable, ShareICT, which will be used in the next phase. The model fit worsened slightly but was still within the good fit range with a CFI of .99 for both countries. The RMSEA value was .03 for Finland and .04 for Bulgaria. With the added latent factors, the model for both countries explained more variance in the science scores (Bulgaria 15.4% and Finland 4.8%).
Phase 4. Finally, as mentioned before, a latent variable, ShareICT, was added to account for the shared variance of the observed predictor variables. The data from both countries were combined into the bifactor model to be analyzed. Because there were no significant declines in model fit, all of the observed ICT variables were constrained to give the model strong invariance and to ensure model convergence (Kline, 2015). The number of factors and correspondence were the same for both countries. The unstandardized intercepts and pattern coefficients were constrained. Having a model with strong invariance ensures that respondents with the same response on an item will also have the same level of ICT measured by that item. With the new multi-group bifactor model, the global model fit improved [Chi-Squared (313) = 1956.23, p < .001] and the explained variance increased for Bulgaria but dropped for Finland. The CFI and Tucker-Lewis Index (TLI) remained at .99, the RMSEA value lowered to .03, and the Standardized Root Mean Square Residual (SRMR) value was .04. The new model accounts for 22.2% of the variation in Bulgarian science scores and 5.3% in Finland. In both countries, greater ICT use was associated with lower science scores (β = -.34 and -.10 for Bulgarian and Finland, respectively). High ICT comfort was associated with higher science scores (β = .18 and .20 for Bulgarian and Finland respectively). Finally, higher ICT availability was associated with lower science scores for students in Bulgaria (β = -.08), but was insignificant for Finland. In addition, ESCS was added to the Phase 4 model as a control variable. Including this control caused slight shifts in the regression coefficients, model fit, and percent of science score variance accounted for in each model. ICTUSE was -.11 for Finland and -.28 for Bulgaria. ICTCOMF was .18 for Finland and .12 for Bulgaria. ICTAVB was -.07 for Finland and -.20 for Bulgaria. The ESCS coefficients with the dependent variable were .34 for Finland and .41 for Bulgaria. The beta coefficient represents the degree of change in the outcome variable for every one-unit of standard deviation change in the predictor variable. As for model fit, CFI and TLI were both .99. RMSEA was .03 while SRMR was .05. Table 5 shows the final model results for Phase 4. Thus, the table indicates that, for example, for every increase of one standard deviation in ICTUSE, the student's Science score changes by -.28 standard deviations of Science in Bulgaria or by -.11 in Finland.

DISCUSSION
The results reveal how the ICT variables measured by the PISA 2015 assessment are related to the science scores of 15-year-old students in a digital frontrunner and in a digital challenger country, respectively. The proposed model also accounted for students' socioeconomic status. The findings suggest that there are both helpful and harmful possibilities for ICT in relation to education. In both countries, a student's perceived competence and autonomy, use in conversation, and ICT interest were positively associated with higher science scores, while the use and availability of ICT technologies at home and at school for schoolwork or entertainment -.28 *** .12 *** -.20 *** Finland -.11 *** .18 *** -.07 *** *p < .05, **p < .01, ***p < .001 were found to have a negative association with science scores. The current findings are supported by both social-cognitive theory (Bandura, 1986) and selfdetermination theory (Ryan & Deci, 2000) that emphasize the relationships among performance and individuals' sense of autonomy, relatedness, competence, and confidence in their own abilities to successfully perform a behavior. Indeed, researchers have associated a range of affordances for ICT use in science education, with ICT promoting science innovations in four major ways: cognitive acceleration, range of experience, self-management, and data collection and presentation (Webb, 2005). Moreover, the physical attributes of the learning environment could alter students' sense of autonomy in relation to ICT (Zandvliet, 2012). In essence, teachers need to gain a deeper understanding of their learning environments as well as of students' self-regulated learning processes when using ICT to support their students' science learning.
An explanation for the association between ICT use and availability with lower science scores is that perhaps students are being distracted by the ICT in terms of the types of activities performed or the time taken to engage in extracurricular activities at the expense of academic learning. It could also be that they may be using technology for its own sake, rather than to support and uplift their learning (Martin-Perpiñá, Viñas i Poch, & Malo Cerrato, 2019; Naumann, 2015). Teaching for innovation is an intentional process and teacher preparation cannot be compensated by a technologically-rich learning environment. It was found that teachers who embed technology in their instruction reinforce their pre-existing practices (Cuban, 2001). Finally, it could be that the pervasiveness of ICT precludes students from engaging in deeper problemsolving activities, as they can find most answers readily available online through web searches or by posting questions on social media. These results are in concordance with findings showing that ICT in a middle-school classroom restricted rather than promoting inquiry (Waight & Abd-El-Khalick, 2007), as the mere presence of computers detracted from meaning-making activities, focusing students on individual accountability.
Taken together, these results suggest the possibility that if ICT is not meaningfully integrated into students' learning, either at school or at home, it could be in fact hindering their science learning. Information about teachers' lesson plans and pedagogy was not available in the PISA data but it could shed more light onto these findings. More information on students' and teachers' views on technology within their particular learning environments may shed more light onto their perceived effectiveness for science learning. Interestingly, the relationship between science scores and available ICT for Finnish students was not significant before controlling for SES, however, it became negatively significant after controlling for SES. The same pattern of results after controlling for SES between Finland, a digital frontrunner, where the majority of students have wide access to a variety of technologies and Bulgaria, a digital challenger, where most students do not have as many devices at their fingertips, indicates that the "digital divide" encompassing the global (Internet access gap between industrialized and developing societies), social (information gap between a nation's rich and poor), and democratic (digital resource use gap in civic engagement) divide (Norris, 2001, p. 1) may not be a factor when it comes to students' science performance. Although they are different in terms of ICT, Bulgaria and Finland are two countries that belong to the European Union, where many programs support teaching and learning in the digital era. Also, young people are nowadays well anchored in social media around the world, actively participating in society as well as producing and consuming information. Perhaps changing the way available technology is used to teach science may yield different results in future PISA administrations. Targeted pedagogy with technology programs offered by the European Union could contribute to diminishing the deleterious effects of ICT use and availability on students' science performance. It is also possible that these results could change when controlling for some of the demographic variables collected via the PISA assessment and that factors such as ICT autonomy and interest could be culturally mediated (Zandvliet, 2012). Future research should focus on the role of covariates such as gender or the percentage of urban versus rural populations in each country.
These results support the findings of some previous research and contradict others. Our results are similar to those of Hu et al. (2018) who showed that the use of ICT at home and at school negatively predict science scores, ICT availability held a negative relationship with science scores, and ICT interest, competence, and autonomy were positively associated with science scores. Our results differ for ENTUSE and SOIAICT. For instance, Petko et al. (2017) studied the impact of ICT use on science scores and our results generally support their findings other than top-performing countries with students who use more ICT at home also have higher scores. We would have expected a positive relationship between ICT use at home and science in Finland but not in Bulgaria but, instead, our results showed a negative relationship in both countries. This study was similar to those of Bulut and Cutumisu (2018), Meng et al. (2018), and Luu and Freeman (2011) in that only two countries were used in the analysis. There were no large-scale differences between the two target countries in the current study other than varying strengths of the predicting variables or the percentage of science score variance accounted for by the model. More disparity between target countries was seen in the other studies, hopefully this can be addressed by including more countries in future studies. Similar to Bulut and Cutumisu (2018), the present study found no relationship for Finland science scores and availability, as well as no benefit of ICT use to science scores for the most part. Meng et al. (2018) reported similar results for autonomy but results were fairly mixed for all the other measures.
As ICT has become ubiquitous in the classrooms and in students' homes, the present study contributes to clarifying the role of several ICT variables to uncover any of their positive effects on students' science learning. This endeavor is important, as findings signal that ICT is currently not used to its full potential for learning and innovation, despite becoming essential in every sector of the economy of the 21st century.

Limitations
One limitation of this study is the lack of experimental control over the variables included in the study. Despite the more advanced methods, this study is correlational. Therefore, causality cannot be determined. It is possible that particular ICT habits cause a change in students' science scores, but it is also possible that having a certain science score drives interactions with ICT.
As the ICT Familiarity Questionnaire (OECD, 2014b) is based on self-reported data, true scores may not be accurately represented in the data. Also, this study is limited by the items on the ICT questionnaire, by the particular science achievement measures employed in the PISA study, and possibly, by the different number of sub-items in each ICT broader question. Using this dataset, we cannot draw definitive conclusions regarding the quality or meaningful use of ICT. Future studies will focus on research involving adolescents, exploring the relationship between ICT and science in more ecologically-valid settings.
Results also showed a discrepancy in the percentage of explained variance between Bulgaria and Finland, with the model explaining science scores much better in the former than in the latter. It is possible that as Bulgaria transitions to becoming a digital frontrunner, the role of ICT in explaining students' science performance will diminish as it has in Finland.
We have abstracted out the nine ICT variables into three broad groups (use, availability, and comfort). However, our results may differ from others in the literature if analyses are based on the sub-scales instead (e.g., the ICT use group represents ICT use at home for schoolwork, for entertainment, and ICT use at school) or on all 15 items of the ICT test. Additionally, our use of the ShareICT variable could introduce overfitting issues, which will be further explored in future work.
A ceiling effect relating to Finland's ICT availability may also obfuscate the true results because the questionnaire is unable to accurately measure availability in countries who are digital frontrunners. Finally, only two countries were used in the current study, which provides a limited insight into the relationship of ICT and science achievement in the scope of different cultural environments. Future studies should examine and employ other multiple-group factor analysis methods such as the alignment method to simplify the measurement invariance analysis for many groups (Asparouhov & Muthén, 2014).

CONCLUSIONS
The goal of this study was to test the effects of ICT variables on students' science performance in two countries from PISA 2015. The findings revealed both positive (ICT comfort) and negative (ICT use and availability) relationships between ICT and science education in two countries with contrasting digital profiles, Bulgaria and Finland, using SEM in PISA 2015. Thus, findings provide an emphasis on the discrepant factors amenable to interventions that may equalize science performance across the two countries. In the past, very few studies have connected all nine ICT variables that were included in this study to science performance. So far, only Hu et al. (2018) and Juhaňák et al. (2018) compared science scores based on all nine forms of ICT. However, neither study employed SEM, which makes the contributions of this paper unique and valuable. Taken together, the results suggest that more in-depth research needs to be conducted into factors that influence students' science performance. Also, there is potential for improving student outcomes under the right ICT conditions. Once we are able to understand the intricate relationship of students' ICT and their academic achievement, we can create tailored changes to how technology is used in schools, aiming to minimize the undesirable effects and maximize the potential helpful aspects of ICT use in education.