Developing a Web-based Assessment System for Evaluating Examinee ’ s Understanding of the Procedure of Scientific Experiments

This research tries to use multimedia animation to design examination questions and develop a Web-based Performance Assessment system (WPA system) to evaluate examinees’ understanding of the procedure of scientific experiments. WPA system also documents and scores the process an examinee responds to multimedia animation questions, including the tracks movable objects leave on the screen, and information examinees input and whether their answers are correct. Examinees’ recorded process of the operation of each multimedia animation question can be replayed for teachers to grasp a full picture of examinees’ process of answering. WPA system can be used for improving and evaluating students’ understanding of the procedure of scientific experiments.


INTRODUCTION
Online assessment has been extensively applied in the field of education, and is gradually gaining its importance in teaching and research (Wang, 2010;Wang, Chiu, Lin, & Chou, 2013).Its primary merits lie in the decrease of paper consumption and fast collection and analysis of data; furthermore, electronic examination questions and papers are beneficial to the exchange, filing and modification of sources of examination questions.Those advantages display milestone significance to teachers and educational researchers.Online assessment, in addition to the aforementioned virtues, also provides machine scoring function, offering immediate examination correction and enabling examinees to receive instant information regarding examination answers and results.This feature gives weight to online assessment in terms of offering examinees timely feedback, lightening teachers' teaching loads, facilitating e-Learning and fostering students' ability of self-assessment (Wang, 2011a;Wang, 2011b).
Science laboratory instruction plays an important role in science teaching.An abundance of opportunities to conduct scientific experiments is the key to introducing core scientific concepts, procedure knowledge, and process skills (Bybee, 2000;Hofstein & Lunetta, 2004).Lunetta, Hofstein and Clough (2007) pointed out that there are four important stages in science laboratory instruction, including planning and design, performance, analysis and interpretation, and application.Planning and design means to organize research questions, predict results, generate hypothesis for examination, and design experimental procedures.Performance means to investigate, to operate and handle instrument equipment and experiment materials, to decide investigation methods, and to organize and record observed data.Analysis and interpretation means to handle collected data and explain data relevance, to develop and generate research findings, to discuss accuracy and limitation of data and experimental procedures, and to generate new research questions based on experiment results.Application means to construct prediction models, to generate hypothesis based on research results, to apply experiment operation skills to other new experiment scenarios and to consolidate assertions.Lunetta et al. further pointed out that these stages all include important procedural knowledge such as understanding and developing experimental investigation design, developing and confirming correctness of procedure and results, and etc.These are all highly related to student's development and understanding of scientific concept.Students can use scientific methods and scientific procedural knowledge to investigate scientific phenomenon and solve problems, and in turn facilitate their own understanding of scientific concepts, models and theories; they can also obtain understanding of science and methods of scientific inquiry (Hofstein & Lunetta, 2004).Lunetta et al. and Kempa (1986) pointed out that the aforementioned four stages can be used as the scaffold for assessing the effectiveness of science laboratory instruction.It can be used to develop assessment mechanism and strategies.Lunetta (1998) and Hofstein and Lunetta (2004) concluded that the key ways to evaluate the effectiveness of science laboratory instruction include practical examinations, laboratory reports, portfolios, continuous assessment and combinations.Practical examinations are to grade learner's actual actions during the experimental procedures; laboratory reports are to grade learner's records during the experimental procedures; portfolios are the documented data learners prepare and collect during the experimental investigation process, which are often collected focusing on one topic or in one semester.The contents include "investigated topics and design of investigation process," "adopted investigation process and observation methods," "findings and explanations," "limitation of findings and new questions."Learners need to collect data about these topics to prove how they complete these procedures, their thoughts and various information they search and collect.Continuous assessment is a type of dynamic assessment.Based on pre-set criteria, evaluators observe each learner's performance during the regular laboratory session and grade them.Combinations is to combine different assessment methods described above.For example, Hofstein, Shore, & Kipnis (2004) integrated continuous assessment and laboratory reports; teachers observe individual students or teams based on the pre-set criteria and evaluate the hot-reports they regularly submit.Hot-reports are mainly used to analyze learner's experiences, observations, data analysis, inference, questions, hypothesis, and based on the investigation results, new research questions are raised and ways to investigate solutions are planned (Lunetta et al., 2007).
All of the assessment strategies have their limitations.That is the reason why assessing the effectiveness of science laboratory instruction is pretty challenging.Rounded and proper assessment for science laboratory instruction is hard to carry out since execution of teaching activities, classroom management, and laboratory safety all need to be taken care of during laboratory instruction.Although it is challenging to perform assessment during science laboratory instruction, Makransky, Thisgaard, & Gadegaard (2016) pointed out that to achieve effective learning in science laboratory instruction, there are some fundamental requirements to meet, including basic knowledge about working in laboratories, basic understanding about how to use different equipment, and basic knowledge about experiment tasks.Makransky et al. believed that without these knowledge and understanding, learners would waste time and cognitive resources and in turn have less learning effectiveness.In other words, to understand learner's learning effectiveness in science laboratory instruction, grading from these aspects can provide some valuable references.This research refers to Makransky et al. and develops online assessment mechanism, expecting to serve as one of the tools for teachers to assess learner's learning effectiveness in science laboratory instruction.
In addition to conducting scientific experiments in a real laboratory, Heradio et al. (2016) and Triona and Klahr (2003) suggested that information communication technology (ICT) is effective in assisting science laboratory instruction.They contended that science laboratory instruction assisted by ICT provides the advantages of convenience, safety, cost-effectiveness, error minimization, time-saving, flexible space, rapidity, and dynamic data display (Heradio et al., 2016;Triona & Klahr, 2003).The 'virtual laboratory' (e.g.Dalgarno, Bishop, Adlong, & Bedgood, 2009;Jagodziński & Wolski, 2015) is a pre-set ideal environment where scientific experiment results are derived from the computing of computer programs.It helps reduce the limitations of scientific experiment classes at school because it can simulate different scientific experiment scenarios and in turn dynamically present the process of scientific experiments via computing (Yang & Heh, 2007;de Grove, van Looy, Neys, & Jansz, 2012).Studies indicated that the virtual laboratory makes students actively participate in experiments and students can learn based on their personal learning paces (Sanger & Greenbowe, 2000;Stieff & Wilensky, 2003;Pekdag, 2010).Some studies also show that before making real scientific experiments in a laboratory, learners can learn first in a virtual laboratory.They can achieve better learning effectiveness (Abdulwahed & Nagy, 2011).However, current virtual laboratory mainly provide the features of virtual operation and simulation, which allows users to perform online scientific experiments but lacks designs of online assessment mechanism to help teachers perform assessment.Therefore, teachers fail to plan science laboratory instruction based on learners' entry behaviours which

Contribution of this paper to the literature
• Most current online testing systems do not have the full functions of recording and scoring examinee's trajectory of moving objects and keyboard inputs on the screen of each animation question.
• Most current online testing systems which appear inadequate in evaluating examinees' understanding of the procedure of scientific experiments.
• This research proposes a design method to develop an online testing which is suitable for evaluating students' understanding of the procedure of scientific experiments.are evaluated by their performance in online scientific experiments.In addition, development of a virtual laboratory takes much time and money and requires advanced technical skills of teachers if they would like to develop themselves.This research proposes another method of ICT-assisted science laboratory instruction, providing teachers a development interface through which teachers can develop questions based on multimedia animation technology.These questions are key steps or concepts in scientific experiments.Teachers do not need to care too much about whether the designs and planning of online experiment environment context are complete or not.They only need to develop questions focusing on specific key steps and concepts.However, with multiple combined questions, learners can make an online scientific experiment by answering several consecutive questions.Teachers are able to evaluate examinee's understanding of the procedure of scientific experiments and further plan and design more effective teaching activities in a real laboratory based on the assessment information.
Current online assessments with machine scoring function focus primarily on multiple-choice questions.These online assessment systems also do not have the full functions of recording and scoring examinee's trajectory of moving objects and keyboard inputs on the screen of each animation question.They are not suitable for evaluating student understanding of the procedure of scientific experiments.However, interactive multimedia teaching materials designed with multimedia animation has been widely put in use for instruction and learning (Bowman, 2012).According to Dancy and Beichner (2006), Wu, Yeh, and Chang (2010) and Wouters, Paas and van Merrienboer (2008), animation could create situational context, reify abstract concepts, describe object's exterior and encourage students' learning motivation.Quality of present examination will be greatly improved insofar as those features of animation are to be implemented in the development of questions.In fact, current global examinations including TIMSS (Trends in International Mathematics and Science Study) and PISA (The Programme for International Student Assessment) are progressively adopting examination questions designed with multimedia animation (Wu et al., 2010).Such examinations virtualize contextual background of reality and incorporate a series of small questions in the item designed based on the animation, forming a situation where examinee simulates solving problems as if in reality by moving and clicking objects and inputting data.In this vein, this research attempts to employ animation in constructing questions to assess examinees' understanding of the procedure of scientific experiments.
Based on above, this research proposes a method and develops an assessment system, Web-based Performance Assessment system (WPA system).The WPA system is developed to manage and administer the multimedia animation questions (MA question), constructed by HTML 5.0 or Adobe Flash TM animation.Moreover, the WPA system further records the process of examinee responses to the MA questions, including tracking the movable objects left on the screen and keyboard inputs, and whether their responses are correct.Those archives allow examinees and teachers to replay the examinee process of answering MA questions.This also allows examinees and teachers to be presented with a full picture of any given MA question taken by any examinee, so that they can achieve the aim of documenting the complete process of conducting a scientific experiment, and attain the optimal outcome of evaluating examinee's understanding of the procedure of scientific experiments.

WEB-BASED PERFORMANCE ASSESSMENT SYSTEM (WPA SYSTEM)
Based on the literature review above and requirements of middle school and elementary school science laboratory instruction, this research brings up an assessment model and develops the WPA system to assist teachers to evaluate examinee's understanding of the procedure of scientific experiments.In this way, teachers can plan more appropriate teaching activities before performing science laboratory instruction in a real laboratory based on assessment information.In the science laboratory instruction of middle school and elementary school, each scientific experiment often has clear procedures and key scientific concepts covered.According to Makransky et al. (2016), learner's learning effectiveness in science laboratory instruction can be influenced by their basic knowledge about working in laboratories, basic understanding about how to use different equipment, and basic knowledge about experiment tasks.Critical features of the experimental procedures are the most essential.Therefore, it is beneficial to effectively understand learner's learning effectiveness by dividing the scientific experimental procedures into steps and focus on matters requiring attention, rules, and critical features in the experiment to design questions and grade.MA questions are designed based on the above principles.By developing MA questions in the WPA system, questions can be developed based on some specific important procedures or concepts.However, with multiple combined questions, learners can make online scientific experiments by answering several consecutive questions and teachers are able to evaluate examinee's understanding of the procedure of scientific experiments at the same time.The WPA system provides two interfaces for teachers and examinees.The system allows teachers to upload MA questions, manage the MA question item bank, and to assemble and administer examinations online.It also allows teachers to oversee the process of examinee reaction to each MA question.For examinees, the system allows them to log in to take the examination, and review the records of all the examinations they have taken, including the scores on each examination and the operational history for each MA question.The six major modules in the WPA system are introduced in Figure 1.

Scoring Module
The Scoring Module enables examinees, after they log in, to take examinations.It also automatically scores examinee performance and documents the process by which examinees respond to each MA question.The scoring module operates with a "Model-View-Controller architecture (MVC architecture)," in which the "View" denotes examinee responses to MA questions exhibited in the browser.The tracks examinees leave by moving mobile objects included in the MA questions, the information entered by examinees using keyboards, and examinee answers, are scored, coded, recorded and transmitted by means of PHP (Hypertext Preprocessor) programs (the "Controller") to the database server (the "Model") (Figure 2a).
While the sequence order is an important feature in scientific experiments, it is more appropriate in developing each MA question for this research to segment the operating procedure of a scientific experiment into various "scoring steps" in accordance with their priorities.Each scoring step of each MA question can be assigned with different point proportion based on its importance.In other words, each independent scoring step comprises one important operating procedure, and all scoring steps form the entire comprehensive operating procedure of the scientific experiment.Based on those scoring steps, the scoring module scores and documents the examinee movements and keyboard inputs on the screen and records the scores of each scoring step (Figure 2b).Examinees can make notes on questions they are uncertain about during the process of the examination for the purpose of returning to those questions after they finish other questions.Examinees are also free to re-do any of the questions before the end of examination, and the system will request examinees to provide their level of confidence for such answers after they finish answering each question for instructor reference (Figure 2c).Step A: A student chooses a proper material in the "Material Zone" and move them to the leaves in the "Working Zone" above.Then the operation and grading of this scoring step is completed.Click "NEXT" to operate on the next scoring step.Step B: A student chooses proper materials in the "Material Zone" and move them to the "Working Zone" to compose the method to soften leaves.Then the operation and grading of the scoring step is completed.Click "NEXT" to operate on the next scoring step.Step C: A student chooses proper materials in the "Material Zone" and move them to the "Working Zone" to compose the actions which need to be done in this step.Then the operation and grading of the scoring step is completed.Click "NEXT" to operate on the next scoring step.

Personal Performance Management Module (PPM module)
In the same manner, the PPM module works on the "MVC architecture" for examinee reviews of their past answering history for each MA question (Figure 3a).In Figure 3b, PHP programs (the "Controller") read from database server (the "Model") the coded data of a given examinee movements on the screen, the information he/she entered with the keyboard, and his/her answers and scores, and transform the data into XML (Extensible Markup Language) format.The data is then resolved into a format that PPM module can accurately read and MA questions then presents the above actions in the browser (the "View").means that the answer of this question has been submitted.C: Examinee is answering MA question number 1; before submitting the answer, the examinee is required to provide his/her level of confidence for such answer.D: Screen of the examinee answering a MA question (a scientific experiment).E: Examinee can click 'Make note' to make notes on the MA question he/she is responding to, and proceed to other questions first before returning to this one.Click 'Submit' to submit the answer and examinees will not be allowed to alter the answer once 'Submit' is clicked.

Item Bank Management Module (IBM module)
The IBM module allows teachers to manage the item bank of the WPA system via a learning concept coding system (LCCS) (Wang, 2014).The LCCS is the dendritic structure of learning concepts for scientific experiments developed by senior teachers, and all MA questions in WPA system are sorted and grouped in accordance with the LCCS.Each MA question thus comes under at least one learning concept classification.

Examination Assembly Module (EA module)
The EA module lets teachers assemble examinations online.Before assembling examinations online, teachers should construct a two-way specification table.For the table they need to determine which concept(s) for evaluation are taken from the LCCS in IBM module, and configure the needed number of MA questions to deliver the chosen concept(s).Teachers will thus select MA questions from the item bank in the IBM module according to the twoway specification table to assemble the examination, and allocate each MA question its proportional score.Teachers can set points for each MA question but the limit of the full score for each examination would still be 100 points.After setting points for each item, system would automatically distribute points for each scoring step of each MA question based on its pre-assigned point proportion.Teachers can also input administrative information after  assembling the examination, including examination title, instructions for the examination, examination time, order of MA questions, whether it is a shared examination, and so on (Figure 4).

Examination Management Module (EM module)
The EM module allows teachers to manage and preview the examinations they have assembled.For unadministered examinations, teachers can re-edit the contents and administration methods, while for administered examinations, teachers can skim through the scores of each examinee, or browse each examinee's trajectory of  .MA questions that match the prerequisite conditions (concepts and Bloom's cognitive taxonomy) will be listed by clicking the number.C: Selected MA questions according to the two-way specification table.D: Administrative information of examination, including examination title, range, time, order of questions, whether this is a shared examination.
moving objects in the MA questions, examinee keyboard processes during the test, and examinee scores on each scoring step (Figure 3a).

User Information Module (UI module)
The UI module is the user management component of the WPA system that administers user's individual information, limits of authority, and passwords for teachers and examinees.Users are able to modify their individual information and passwords with this module, and teachers are able to create accounts for examinees and put them into groups.The group data provides the EA module with the basis for configuring an examinee roster for each examination.

CONCLUDING REMARKS AND FUTURE DEVELOPMENT
This research develops the WPA system and proposes an ICT-assisted science laboratory instruction strategy based on middle and elementary school science curriculum.In the WPA system, ICT is applied to develop several multimedia animation questions for some important procedures or concepts of a scientific experiment.Then by combining these questions, teachers allow learners to answer several consecutive questions in the WPA system and make scientific experiments online.In this way, teachers can also evaluate examinee's understanding of the procedure of scientific experiments.According to Abdulwahed and Nagy (2011), learners achieve better learning effectiveness if they learn through technology-enhanced scientific experiments before making scientific experiments in a real laboratory.The WPA system in this research not only allows learners to perform online scientific experiments by answering MA questions but supports assessment.Based on assessment results, teachers can plan and design follow-up teaching activities in a real laboratory to enhance student learning effectiveness.It is also anticipated that learners can take advantage of the WPA system to facilitate their learning of scientific experiments.Based on Zacharia (2007), Wiesner and Lan (2004), and Abdulwahed and Nagy (2009), by participating in online scientific experiments, learners obtain complementary learning experiences and the effectiveness of science laboratory instruction in a real laboratory can be improved.Moreover, online scientific experiments provide learners a scaffold with which they can more easily understand key procedures and concepts during a scientific experiment (Honey & Hilton, 2011).In addition, WPA system documents and scores the process by which an examinee responds to MA questions, including examinee movements and keyboard inputs for each MA question.The records can be replayed after the examination is completed, to give teachers a full grasp of examinee processes for taking the examination.Therefore, teachers can learn what obstacles students face in conducting scientific experiments, and perform remedial teaching for students or improve their own teaching.Examinees can also perform self-assessment by taking the examinations administered in the WPA system and review their operational history for each MA question, and reflect on and enhance their operation of the scientific experiments.
This research brings up a strategy about leveraging ICT to assist science laboratory instruction and assessment.Lunetta et al. (2007) concluded that there are many ways to assess the effectiveness of science laboratory instruction.However, these methods are limited to teacher's actual teaching environment and resources, which makes the execution of these assessment strategies challenging.Makransky et al. pointed out that learner's learning effectiveness in science laboratory instruction is mainly influenced by their basic knowledge about working in laboratories, basic understanding about how to use different equipment, and basic knowledge about experiment tasks.Makransky et al. also suggested paying special attention to the critical features of the experimental procedures.Based on this, development of MA questions in this research focuses on matters requiring attention, rules, key steps, and critical features in the experiment.The whole scientific experiment is divided into many small items and teachers are allowed to adopt multimedia animation technology to develop MA questions online.The WPA system is used to administer online assessment and record and grade learner's answering history.The WPA system and MA questions should be able to serve as assessment strategies for science laboratory instruction and be leveraged by teachers to enhance their teaching effectiveness.It is suggested that it is necessary to conduct further researches to prove the effectiveness of the WPA system in assisting science laboratory instruction.In addition, the current WPA system requires a learner feedback mechanism so that learners can write self-reflection information while viewing their own answering history.The information also serves as reference for teachers to facilitate learner's self-reflection on their own online scientific experiment.In composing an exam, a mechanism should also be developed so that teachers can select items based on their difficulty levels and the examination can be composed with proper difficulty.In addition, in the WPA system, constructing MA questions poses technical difficulties for teachers with its interactive design.Presently a function library on the interactive features of MA questions has been assembled, composed of about fifty functions.For teachers equipped with basic knowledge of Web page design and animation program, a few hours of training will facilitate them in using the interactive function library to design questions, as well as administrate examinations using the WPA system.The future perspective of the WPA system dictates the continuous development of functions of various kinds based on the needs of MA questions along with increased ease of use for the interactive design.Training workshops for teachers to teach the functions of WPA system shall be held, both to enhance teachers' technical knowledge of the system and to encourage them to develop MA questions on their own.Online MA questions for scientific experiments will be even more richly diversified to create robust support for students and teachers.

Figure 1 .
Figure 1.System architecture of WPA system

Figure 2c .
Figure 2c.Screen of an examinee taking an examination in the Scoring Module.A: MA question number; the examinee can choose to answer other questions first by clicking the number.B: Answering status; the 'V' means that the answer of this question has been submitted.C: Examinee is answering MA question number 1; before submitting the answer, the examinee is required to provide his/her level of confidence for such answer.D: Screen of the examinee answering a MA question (a scientific experiment).E: Examinee can click 'Make note' to make notes on the MA question he/she is responding to, and proceed to other questions first before returning to this one.Click 'Submit' to submit the answer and examinees will not be allowed to alter the answer once 'Submit' is clicked.

Figure 3a .
Figure 3a.Browsing the past examination record of an examinee (stu1).A: Administrative information of examination, including downloading records, administering time of the examination, examinee's ID, examination title and scores.B: 20 points for MA question number 1, and stu1's level of confidence for answering this question is 80%.C: MA question number 1 comprises of two scoring steps: 1-1 and 1-2, respectively.The examinee scores both scoring steps to receive 10 points each.D: Stu1 clicks on the number of scoring step (i.e.1-1 or 1-2) in C section to replay the process and tracks of how he/she responded to such scoring step.

Figure 3b .
Figure 3b.Model-View-Controller architecture in the PPM module

Figure 4 .
Figure 4. Screen of assembling an examination.A: Function menu of teacher's interface.B: Construction of a two-way specification table.MA questions that match the prerequisite conditions (concepts and Bloom's cognitive taxonomy) will be listed by clicking the number.C: Selected MA questions according to the two-way specification table.D: Administrative information of examination, including examination title, range, time, order of questions, whether this is a shared examination.