Outcomes of a Drug Dosage Calculation Training Smartphone App on Learning Achievement , Metacognition , and Flow State According to Prior Knowledge

The purpose of this study was to define the effectiveness of a smartphone-based dosage calculation training app on nursing students’ learning achievement, metacognition, and flow based on prior knowledge. The study used a quasiexperimental design with a preand post-test. Participants were 157 nursing students from 3 universities with baccalaureate programs in South Korea. After recruiting the experimental and control groups, they were also categorized by prior knowledge of drug dosage calculation ability into above and below-mean clusters. The experimental groups were provided with the smartphone-based app. In the above-mean cluster, changes in total learning achievement (Z=3.16, p=.002), drop rate calculation (Z=2.76, p=.006), metacognition (Z=2.50, p=.012), and flow score (Z=2.42, p=.016) in the experimental group were significantly higher than those in the control group. A smartphone-based calculation training app improves calculation achievement, metacognition, and flow among students with higher prior knowledge. For learners with lower prior knowledge, additional instructional design or implementation strategies may be needed.


INTRODUCTION
Medication administration is considered to be one of the most important responsibilities of nurses, for whom dosage calculation competence is a crucial skill, because medication administration errors related to wrong dosage committed by registered nurses are common (Fathi et al., 2017).Nursing students showed relatively low dosage calculation competence in several studies (McMullan, Jones, & Lea, 2010;Özyazıcıoğlu et al., 2017).Further, the mean score of drug calculation skill of registered nurses showed unacceptable results at 60%, and about half of them failed basic numerical skill tests (McMullan et al., 2010).This deficiency may be from insufficient mathematical knowledge and understanding of the correct medication formula for each situation (Coyne, Needham, & Rands, 2013).Therefore, different teaching and learning strategies for dosage calculation education have been developed such as e-learning (Simonsen, Daehlin, Johansson, & Farup, 2014), or mobile technologybased strategies (Liu, Lin, Tsai, & Paas, 2011).
Based on the previous study, mobile technology such as smartphones is considered to be an innovative teaching strategy in both traditional classrooms and learning outside the classroom (Liu et al., 2011), that can be realized due to individualized interfaces, portability, real-time access to information, and feedback.According to the metaanalysis of the use of mobile devices on learning performance with teaching and learning (Sung, Chang, & Liu, 2016), its effect in education of professional subjects or mathematics had a medium effect size.Although several smartphone-based learning apps have been used as educational supplements in healthcare, the majority of research has focused on lower-level knowledge and skill (Mosa, Yoo, & Sheets, 2012), not higher-level knowledge like calculation and evaluation.In addition, most previous studies in this area have examined the effects of smartphone-based educational programs on learning achievement (Sung et al., 2016), overlooking metacognition and flow as procedural outcomes.
Metacognition is called "cognition of cognition" or "higher cognition," and defined knowledge about cognition and regulation of cognition as two components (Carruthers, 2009).First, knowledge about cognition defined as stable knowledge about the learner's own cognitive system, awareness of the current state of cognition, and appraisal of the significance of thought and memories (Wells, 1995).This means that what students know about themselves, the tasks they conduct, and their learning strategies, are essential for self-directed learning.If learners can immediately recall knowledge after learning or perceive the current status of their knowledge and learning strategy, they are regarded as having high metacognition.Learners with high metacognition tend to think themselves as good learners and develop positive beliefs and attitude about their learning (Graham, 2006).Second, regulation of cognition means supervising and managing one's own learning process and understanding of how to regulate those processes to maximize learning achievement (Wells, 1995).Learners having higher metacognitive skills receive better scores on their exams and complete work more efficiently because they utilize effective tools and modify learning strategies and skills based on knowledge effectiveness (Magno, 2010).Based on the previous studies, there were significant changes in metacognitive perception to achieve autonomous learning after augmented-reality-based learning (Cotterall & Murray, 2009), and higher metacognition induces better learning achievement (Ferrer-Torregrosa et al., 2016).Smartphones provide benefits such as individualized interfaces, portability, and real-time access to information; it is helpful that students can improve their metacognition by managing their own learning "just-in-time" and "when-needed." Flow is understood as a state of optimal experience in which a person derives pleasure from the intrinsic experience of performing a task (Nakamura & Csikszentmihalyi, 2009).Nakamura and Csikszentmihalyi (2009) describe flow as an integrated status of cognitive domain and affective domain; that is, a feeling of enjoyment and immersion, energized focus, and involvement, frequently accompanied by positive affect or pleasure (Hung, Sun, & Yu, 2015).In a flow state, we feel time stand still and lose our sense of self, and enjoy engaging in a performance for its own sake (Kwon, 2008).If learners can concentrate and neglect unrelated thoughts, for example about time, space, or themselves, the learning of flow will occur, leading them to feel pleased and joyful, which is positive for the learning process.Concentration and interest in the activity are needed for flow, and learners may feel flow if they experience enjoyable moments immersed in the joy of learning (Nakamura & Csikszentmihalyi, 2009).According to previous studies, a multimedia-based educational game enhances flow level and high interactivity (Hung et al., 2015), and in particular attracts learners' interests and attention compared to the non-game environment (Sitzmann, 2011).
When developing a dosage calculation educational program for enhancing the calculation skills, it is difficult to address disparate learning needs due to the diverse levels of students (Hutchinson, Mitchell, & St John, 2011).Entry points need to be considered to ensure consistency among students for a similar level of calculation skill.However, it is not easy to find research comparing educational programs' effects by prior knowledge level, which based on one study serves as a necessary and sufficient condition for development of learners' interest in continuing to learn further (Hailikari, Katajavuori, & Lindblom-Ylanne, 2008).Even when the educational techniques in use are well developed, learning performance cannot be accomplished by learners who have lower prior knowledge.Prior knowledge has been considered to be the variable predicting the largest proportion of achievement, and also predicts part of flow (Kang, Kim, Kim, Park, & Koo, 2009).Based on these known relationships, the aim of this study is to answer the research question, "Are there differences in the effectiveness of a smartphone-based dosage calculation training program on nursing students' learning achievement, metacognition, and flow based on prior knowledge?"

Contribution of this paper to the literature
• This study highlights the necessity for examining not only learning achievement but also prior knowledge and procedural variables such as metacognition, flow when evaluating the effectiveness of a smartphonebased learning app.
• The results show that the drug dosage calculation training smartphone-based app was more effective on learning achievement, metacognition, and flow in the higher prior knowledge group.This means that additional instructional design of smartphone-apps or implementation strategies by instructors may be needed for learners with lower prior knowledge.
• This study is expected to provide insights for smartphone-based learning app development and selflearning guidance for drug dosage calculation.

Study Design
The study used a quasi-experimental design with a pre-and post-test.

Population and Sample
Participants were second-year nursing students, as this app focused on basic drug dosage calculation.Three universities with baccalaureate nursing programs were recruited in a major city in South Korea.The three nursing schools were selected because of similar entrance scores and curricula for medication dosage calculation.The inclusion criteria were as follows: participants had to be second-year nursing students who 1) had learned detailed drug dosage calculation formulae in their subjects, 2) had no experience with medication administration, 3) were female students, and 4) consented to participate.Male students were excluded to improve participants' homogeneity the proportion of male students was not similar between university.Participants were allocated conveniently into two groups based on their university, to prevent diffusion of the intervention; students from two universities were placed in the experimental group, and those from the third university into the control group.Participants in each group were also classified into two clusters by pre-test learning achievement score: the "belowmean cluster" and the "above -mean cluster."The mean was chosen as the cutoff point because it was the highest score among mean, median, and mode.As the distribution of the participants' prior knowledge showed positive skewness, the mean was higher than the median.For the even allocation of the participants, we determined that the mean was more appropriate as the cutoff point.Moreover, as a higher score is needed for medication patient safety, the higher cutoff point was adopted.Statistically, the mean contains more important information than the median or mode, is easy to understand, and representative of the data.
The sample size was calculated based on Kim, Park, and Park (2012).A power analysis for a t-test was performed using an alpha coefficient of 0.05, beta of 0.20, and effect size of 0.20.The required sample size was 199 participants.Ninety-three nursing students from two universities were recruited as the experimental group and 85 (91.4%) agreed to participate in this study.An additional 97 nursing students from another university, the control group, were recruited, and 82 (84.5%) agreed to participated.The reason for the high voluntary participation rate was considered to be because nursing students had high motivation for and interest in enhancing drug dosage calculation competence.A total of 167 participants was recruited and categorized into the four groups (Figure 1).Eighty-one students (95.3%) in the experimental group and 76 students (92.7%) in the control group completed the pre-and post-test.The effect size of this study was 0.46 with an alpha coefficient of 0.05, a beta of 0.20, and an input of 157 participants.

Development of Intervention
The smartphone-based app for this study was developed by the research team.Analysis of learners and their environment, designs for templates and contents, program development, and evaluation by informatics experts were performed, and the effectiveness and validity were proved in a previous study (Kim et al., 2012).The smartphone-based app adopted a cognitive-load theory and contained two pages; a principle learning page and a practical game page.The former consisted of four subcategories: "metric conversion," "tablet calculation," "fluid dosage calculation," and "drop rate calculation."On this page, the participants could see the solution to the problem they were assigned and use it to comprehend the principles of dosage calculation.The latter contained three chambers, as follows: "basic," "intermediate," and "advanced."In each chamber, 15 questions were shown for each of the four subcategories.Response scores were calculated and appeared as the participants' rankings in charts.The app can provide individualized interfaces, give real-time access to information related to the use of the app and feedback, and maximize strengths as smartphone-related learning material.This app was developed to be used with both the iOS and Android operating systems; participants used it on their own smartphones.

Procedures of Intervention
Researchers developed contents and a handout for a 1 hour lecture on basic mathematics and drug dosage calculation formulas.The developed teaching materials and methods were reviewed by two experts to assess face validity and content validity, and the materials were provided to three different lecturers in each university.To ensure teaching method consistency among lecturers, two discussion times were held.In order to get reliable results, a double-blind trial was used for grouping and providing intervention.The participants were taught metric conversion, formulas for drug dosage (tablet and fluid amount), and infusion and drop rate calculation in the classroom, and after the lecture, completed the pre-test questionnaire.The available test time was 20 minutes, and the participants were allowed to use a calculator.
After the test, a smartphone app using approved login information was provided for the experimental group.In order to increase treatment fidelity, the researchers guided the participants to use the app at least once a day and, once a week, reminded them to use the app after confirming experimental group members' contact time and frequency each week.For the control groups, self-study was recommended using the handout and conventional study method during the same four-week period.Four weeks after starting the intervention, a post-test was administered to both groups, using a similar questionnaire as that used for the pre-test.With ethics in mind, the smartphone app was provided to the control group after the research period.The study was conducted from October 2012 to June 2013.

Learning achievement
In this study, learning achievement is the ability to calculate tablet count, fluid amount, and drop rates exactly.The questionnaire consists of three dimensions (tablet count, fluid amount, and drop rates) with nine items.This scale was validated by a nursing education specialist who had over 10 years of nursing experience and a doctoral degree, using CVI (content validity index); all items scored over 80 percent on validity.Coding was registered as "1" if the respondent selected the correct answer and "0" if the wrong answer.Total scores could thus range from 0 to 9, with a higher score indicating greater learning achievement.To test item difficulty, items answered correctly by more than 80% of the participants were considered too easy and items answered correctly by less than 20% of the participants were considered too difficult based on Sung (2016).The item difficulties of the 18 questions at preand post-test were moderate or low difficulty.The discrimination index may range from -1 to 1, which is less than .20 as "no discriminating power," .21-.40 as "having discriminating power," and more than .41as "high discriminating power," The discrimination of the 18 questions at pre-and post-test ranged from .21 -.40.Kuder-Richardson Formula 20(KR-20) reliability ranged from.76 to .78 at pre-and post-test.

Metacognition
An instrument developed by O'Neil and Abedi (1996) and translated into Korean by Park and Kweon (2010) was used to measure metacognition in this study.It includes 20 items in four subscales: awareness (five items), cognitive strategy (five items), planning (five items), and self-checking (five items), all answered on a five-point Likert scale (from 1, strongly disagree, to 5, strongly agree), with higher scores indicating higher levels of metacognition.The construct validity of the original metacognitive inventory was acceptable (O'Neil & Abedi, 1996).Cronbach's alphas for the original community college sample were from .75 to .79 (O'Neil & Abedi, 1996) and for this study, from .76 to .87.

Flow state
In this study, we used the Flow State Scale to assess participants' degree of concentration or interest when answering calculation questionnaires.It was developed by Jackson and Marsh (1996) in a study of participants in a physical activity setting, and was adapted to answering calculation questionnaires in this study.It consists of 36 questions answered on Likert scale as above.Higher scores indicated higher concentration and higher feeling of control over drug dosage calculation performance.Cronbach's alpha coefficient was .83 in the developmental study (Jackson & Marsh, 1996) and .85 in this study.

Statistical analysis
Data were analyzed using SPSS for Windows, version 23.0 (SPSS, Chicago, IL, USA).Descriptive analyses were used to determine the characteristics of the participants.To examine the homogeneity of the general characteristics and study variables of participants, Fisher exact tests and Mann-Whitney U tests were conducted in above and below-mean clusters, because study variables did not show normal distribution.Mann-Whitney U tests were conducted to compare the differences of outcome variables (learning achievement, metacognition, and flow) between the two groups in each cluster.

Ethical consideration
Prior to data collection, approval for this study was obtained from the Institutional Review Board of our university (IRB No. 1041386-20121005-HR-011-03).During the recruitment period, the purpose, voluntary participation, confidentiality of the information, and procedures of the study were explained to nursing students in each school.Also, the students were told that the results would be considered confidential for study-related purposes and would not have influenced their training path.Written consent was obtained from all of the participants before the pretest.

Homogeneity Test of Research Variables between Groups
General characteristics as potential impacting variables on prior knowledge, including age, gender, and phone use time, were homogeneous.Baseline characteristics of the participants are shown in Table 1.Based on the homogeneity test, there were significant differences in baseline research variables: fluid amount calculation (Z=2.59,p=.010) and drop rate calculation (Z=2.39,p=.017) in the above-mean cluster, total learning achievement (Z=4.62,p<.001), tablet calculation (Z=6.58,p<.001), metacognition (Z=3.93,p <.001) and flow (Z=4.13,p<.001) in the below-mean cluster.The participants were not homogeneous in learning achievement, metacognition, and flow.

Effects of the Intervention on Learning Achievement, Metacognition and Flow
The effects of the intervention on learning achievement, metacognition and flow are shown in Table 2.In the above-mean cluster, changes in total learning achievement (Z=3.16,p=.002) and drop rate calculation (Z=2.76,p=.006) in the experimental group were significantly higher than those in the control group.Total metacognition (Z=2.50, p=.012) including awareness (Z=2.36,p=.018) and planning (Z=2.58,p=.010) of the experimental group changed more significantly than those of the control group.Difference of the flow score (Z=2.42,p=.016) in the experimental group was higher than in the control group.In the below-mean cluster, changes of all outcomes variables in the experimental group were lower than in the control group.

DISCUSSION
A previous study has shown that prior knowledge can influence students' interaction with a technology-based learning environment and affect their use of self-learning strategies in it.It is also known that there are significant differences in the use of metacognitive strategies and the experience of flow between prior knowledge groups (Taub, Azevedo, Bouchet, & Khosravifar, 2014).Therefore, we assumed that not only learning achievement, but also metacognition and flow exhibited during the use of the smartphone-based training program, would vary according to the participants' prior knowledge level.We will discuss the comparison of the differences between pre-and post-tests after separating the participants by their prior knowledge level.
In the above-mean cluster, the learning achievement change between the pre-and post-tests of the experimental group was significantly higher than that of the control group.According to a previous study (Wright, 2008), the most important resource for developing drug calculation skills and abilities is regular exposure to calculation in clinical practice.In the case of nursing students, because we cannot provide a real continuous clinical environment for calculation, a virtual reality environment using a smartphone -based app will be helpful.The present results showed that the smartphone-based app was an effective supplement as a traditional pedagogical agent for learners with higher prior knowledge.Furthermore, because RNs working in a unit that rarely calculated drugs have reduced regular exposure and educational opportunities after entering practice, smartphone-based training programs are expected to be able to play a supportive role, increasing these nurses' drug dosage calculation competence.
There were significant differences in metacognition and flow according to the groups.The metacognition score change of the experimental group was higher than that of the control group, especially for awareness and planning.On the practical game page, students presumably define dosage calculation as a task they must resolve, set the goal based on their own plan and get feedback by their ranking.Repeated testing would produce metacognitive questions, such as "What do I know?", "What do I want to know?", and "What have I learned?"These questions would foster metacognitive power in the mobile learning environment.Based on the fact that the relationships between self-learning and metacognition are recursive (Cotterall & Murray, 2009), repeat use of this app tends to enhance metacognitive power, and highly metacognitive learners tend to understand and manage their learning better.
The flow state score change of the experimental group was also higher than that of the control group.To experience flow state, clear goals, loss of self-consciousness, transformation of time, and unambiguous feedback are also needed (Jackson & Marsh, 1996).In this study, the principle learning page of the app showed clear goals by presenting information on metric conversion, tablet calculation, fluid amount calculation, and drop rate calculation, while the practical game page allowed the participants in the experimental group to experience flow state via loss of self-consciousness.Learners might feel the time passing in a way different from normal, caused by various interactions between technology and user.When actively participating using this page, learners were expected to be motivated by its imagery and auditory feedback, reinforcing their flow state more than that of the control group.Similar to previous work (Hung et al., 2015), it is concluded that game-based learning promoted flow state in this study.
In the below-mean cluster, there were also significant differences in total learning achievement and in subcategories including tablet count, fluid amount, and drop rate calculation.However, the differences of learning achievement in the experimental group were lower than those in the control group.These results are not consistent with those of previous studies.All kinds of educational strategies including classroom teaching were at least somewhat effective (Härkänen, Voutilaine, Turunen, & Vehviläinen-Julkunen, 2016); in particular, the use of simulation in medication calculation has shown positive significant effects on mathematical ability (Grugnetti, Bagnasco, Rosa, & Sasso, 2014).Two possible explanations for these results present themselves: 1) lack of prior knowledge and 2) failure to control learning speed.That is, if learners have a schema related to prior knowledge, that schema will be activated automatically, reducing cognitive load and promoting problem -solving, when the learners encounter new information (Byun, Ryu, & Song, 2011).Learners in the experimental group presumably feel more awkward using the smartphone-based application and calculation questionnaire than do those in the control group; as a result, they may fail to properly control learning speed and may feel more of a cognitive load.In addition, due to these frustrations, learners in the experimental group may stop studying, while the control group continues to use the training book.
A smartphone-based training program itself is not effective for learners' metacognition and flow, even though smartphone-based apps are regarded as a supportive tool for enhancing metacognitive power (Lee, 2013) and flow (Lee & Lee, 2013).We provided a "worked example effect" showing an instructional example of the solution to the problem; this might improve metacognition.Game-based learning content and interactivity help learners concentrate, which is strongly related to the experience of flow (Hung et al., 2015).However, smartphone-based training program usage showed no effect on learners with low prior knowledge.Two explanations might exist.
First, it could be elicited more slowly in participants with poor prior knowledge, due to their lower learning competence using the application.Therefore, until their knowledge schema settles down more concretely (Byun et al., 2011), it might be difficult to rapidly improve their metacognition and flow.Second, as previously written, because learning activities and metacognition are recursive traits (Cotterall & Murray, 2009), unchanged metacognition may become a barrier to improving learning activities and learning achievement, and perhaps also vice versa.To address this, re-verification needs to be performed in a replication study with longer intervention time and more participants.
This study has yielded useful data on the effectiveness of smartphone based training combined with prior knowledge, but it also has some limitations that should be registered.First, this study was not randomized; instead, convenience sampling was used because of the difficulty of obtaining consent from the nursing schools involved.Heterogeneously made teams can foster communication and other social skills such as problem solving and critical thinking (Schmidt, 2007), leading to collaborative study and interactive relationships.Thus, this research should be repeated using this application but with random assignment of participants should be done.

CONCLUSIONS
Effects of the application on learning achievement, metacognition, and flow differed by users' prior knowledge level.Specifically, smartphone-based app users in the higher-prior-knowledge group felt more concentration and interest in dosage calculation and gained better results for learning achievement, metacognition, and flow.For learners with lower prior knowledge, additional instructional design or implementation strategies for enhancing the learning outcomes should be prepared.

Figure 1 .
Figure 1.Flow chart for participants recruitment

Table 1 .
Homogeneity of General Characteristics, Learning Achievement, Metacognition, and Flow by Groups at the Baseline

Table 2 .
Group Comparisons of Learning Achievement, Metacognition, and Flow at the Post-test (N=157)