How do Students Perceived Computerized Feedback as Effective ?

Feedback plays an important role in fostering deep learning. It is widely recognized as one of the most powerful influences on students’ learning. Meanwhile, timeliness is one of the important elements for feedback to be effective. In line with the technology development, the trend of feedback delivery has been shifted from conventional written and oral feedback to computerized feedback. It can be planned and delivered to students in a timely manner. To fully utilize the advantage of computerized feedback, students’ views should be taken into account. This study investigated students’ perception on computerized feedback through semi-structured interviews. From the results, seven themes were identified: (1) Meaning, (2) Content, (3) Comprehensibility, (4) Usefulness, (5) Timeliness, (6) Emotion, and (7) Attention. The findings of this study emphasize the needs for understanding students’ perceptions of computerized feedback to maximize its role in improving students’ performance.


INTRODUCTION
Feedback is an essential element in learning that enables students to raise their level of awareness in relation to their strengths and also identify weaknesses which require further attention.Frequent and immediate feedback is critical for learning (Narciss & Huth, 2004).It is widely recognized as one of the most powerful influences on students' learning (Johnson, Reisslein & Reisslein, 2015).Through feedback, students can reflect back, know about what they are doing well and what do they need to improve.Without feedback, a test may promote misconceptions (Epstein et al., 2002).Timeliness is one of the important elements for feedback to be effective.According to Mutch (2003), it is important to give feedback to students within certain timelines when it is still meaningful to them.Students who received immediate feedback showed higher response identification accuracy, confidence rating and memory retention (Brosvic, Epstein, Cook, & Dihoff, 2005).They appreciate prompt feedback that could reveal misconceptions and convert their mistakes to correct answers (Cotner et al., 2008;Epstein et al., 2002).
With the rapid development of educational technologies, feedback delivered to students is not limited to faceto-face or written feedback.It can be delivered through computer-based testing (CBT).It has become one of the mediums in providing computerized feedback to students due to its ability in providing instant feedback to students.The trend of providing feedback has been shifting from a conventional handwritten way to the electronic version (Chang, Watson, Bakerson, & McGoron, 2013).Students' views on the computerized feedback could help to frame its effectiveness in the teaching and learning process.Meanwhile, every person tends to view things in different ways and with their own perspective.Understanding how students perceive their learning is vital (Tudor, Penlington, & McDowell, 2010).What students perceive from their learning situation will influence their performance and also their motivation in learning.Moreover, the condition for computerized feedback to be effective can be better understood when students' opinions are taken into account.Therefore, this study aims to contribute to build a richer picture of students' view of computerized feedback, by finding out how the computerized feedback is perceived by students.

METHODOLOGY
This study adopted qualitative research method.In this study, sampling was constrained to schools that are having computer lab, sufficient amount of computers and internet connection.School X which fulfilled this criteria and implements tracking system (students are divided into different classes according to their academic achievement) was chosen.Cluster sampling was used to select desired class to be involved.In total, ten Grade 7 students in one class were selected randomly (average age: 13 years old), as follow: An algebraic expression misconceptions test was constructed and transformed into CBT test by using XAMPP and phpMyAdmin.XAMPP is a free and open source cross-platform web server solution stack package.It is designed for use as a development tool for web designer (Apache Friends, 2015).Meanwhile, phpMyAdmin is a free software tool written in PHP, intended to handle the administration of MySQL over the web.
In total, 22 items were constructed by referring to textbooks and reference books published locally.The allocated time for the test is 40 minutes.Computerized feedback was presented to the students after an answer was clicked which comprised of the four major components (Correctness, Information, Reinforce, and Directive guide).Figure 1 and 2 show the example of screenshot of the detailed feedback given.With the feedback provided, students gained information about their answer and proceeded to the next question after that until all 22 items were answered.

Contribution of this paper to the literature
• Different feedbacks might yield different outcomes.This study aimed to understand how do students' perceived computerized feedback as effective which leads for future improvement.
• Feedback has to be delivered to students in a timely manner while it is still meaningful to students.This study also highlighted the advantage of using computer in giving immediate feedback to students.

Semi-structured Interview
Students' perceptions on the feedback were investigated by using semi-structured interviews.A semistructured interview instrument was adapted from Rowe and Wood (20008) to assess students' perceptions towards feedback for several reasons.Firstly, the instrument was designed to explore students' perception of feedback.Secondly, the instrument was designed based on the themes identified and extracted from the qualitative data.Thirdly, Budge (2011) also adapted this instrument in his study to assess students' perception of electronic feedback.The interview instrument comprises of 11 questions and required about 30 minutes to complete.Trustworthiness of the interview instrument was facilitated by the following steps: (a) Member Check -Member check is "taking data and tentative interpretations back to the people whom they were derived and asking them if the results are plausible" (Merriam, 2001, p. 204).After the interviews were transcribed, the transcripts were validated by the students.This can ensure that the findings' resembled students' perception on the feedback designed in this study.
(b) Language Used to Interview the Participants -The language used in the interview between researcher and students are important.Without using language that is mastered by students, communication barrier might happen and the response received might not be complete or accurate.In this study, researcher used Malay and Mandarin language when interviewing the students.
(c) Interrater reliability -Interrater reliability (also known as consistency estimate) is used to assess the degree to which different raters make consistent estimates of the same phenomenon.It is important to examine interrater reliability so that data collected in this study is correct representations of the variable measured.
In this study, semi-structured interview data was recorded by using audio-taping.It was then coded by two raters.Then, Cohen's kappa was used to examine the interrater reliability.From the analysis, there was substantial agreement between the two raters, k = 0.671 (p < 0.001).

Research Procedure
There were two main sessions involved: (a) Session One -Intervention and (b) Session Two -Interview.During the intervention phase, all students took the CBT individually.Computerized feedback was presented after every question was answered, which consists of four elements: (a) Correctness, (b) Information, (c) Reinforce, and (d) Directive guide.Post-interviews were conducted individually by the researcher with the students to determine their perceptions on the feedback presented in Session 2. During the interviews, the researcher took note and audiotaped the interview.Audio-taping is convenient and reliable to ensure that the original data is available at any time (Gay, Mills, & Airasian, 2009).Following audio-taped data collection, the recordings were transcribed for analysis purpose.

RESULTS
In this study, key themes that best summarized the students' perceptions towards the computerized feedback were identified: (1) Meaning, (2) Content, (3) Comprehensibility, (4) Usefulness, (5) Timeliness, (6) Emotion, and (7) Attention.They were then further analyzed and compared evidence from the different data sets.To protect the identity of the interviewees, all names used are pseudonyms.

Meaning
From the interview data, students perceived CBT feedback as: (a) explanation of correct answer, (b) knowledge, and (c) information.

Content
All students agreed that the content of the CBT feedback received was sufficient.There is no extra information should be added in the CBT feedback.In addition, five students expressed further that the CBT feedback presented is informative.Meanwhile, two students further described the content of the CBT feedback as accurate and relevant."satu… pengajaran yang… memberi maklumat yang penting (a… lesson that… gives important information)" (Interview A4).

Comprehensibility
Although most of the students were satisfied with the content of the CBT feedback given, most of them are having comprehensibility problem.
Only two students (Student A1 and A10) felt that the CBT feedback presented is easy to understand.Meanwhile, eight students cannot understand the CBT feedback well.They were having difficulty in understanding the CBT feedback in some degree.They found that it is hard to catch what the CBT feedback meant due to two main factors.Firstly, it is due to the language used.The CBT feedback delivered was presented by using Malay language.This might be a disadvantage for non-native speakers such as Chinese and Indian students.

Usefulness
In the interviews, most of the students viewed CBT feedback as highly important.They repeatedly commented on the usefulness of the CBT feedback delivered.In particular, students described how they benefited from the CBT feedback.In general, there are two categories that arose in this theme: (a) error detection, and (b) error correction.

Error detection
When the students were being asked whether the CBT feedback presented was helpful in reflecting their mistake, eight students agreed (A1, A2, A3, A4, A5, A6, A8, and A10).Student A1 expressed that he knew his mistake after reading the CBT feedback.Generally, he read through the feedback without spending much time on it, but paid more attention if his answer was wrong.By looking on the example solution shown in the feedback, he was able to identify his error (Interview A1).Similarly, Student A2 claimed that the CBT feedback delivered helped her to detect the error did.

Error correction
In line with the use of CBT feedback in error correction, eight students (A1, A2, A3, A4, A5, A6, A8, and A10) perceived that CBT feedback is useful in error correction and improving their performance.For example, Student A1 shared that he read the CBT feedback and learnt the right way to solve the question.When came across to the next question which was similar to the previous question, he was able to get it right.From this, there is one important point has to be highlighted that, error correction is notable in similar questions.This is further ascertained by the experience shared by Student A10.He agreed that the CBT feedback helped him to correct his error in questions which are identical.As another example, Student A4 shared hat CBT feedback gave her better picture on how to answer the question correctly.In turn, she performed better in the next question.

*I= Interviewer; S = Student
However, Student A8 commented that the CBT feedback presented helped him to correct his error and perform better in the next question, but not exclusively.This means that, after reading the feedback, in the next question, sometimes he got correct but sometimes got wrong.

Timeliness
In this study, the CBT feedback was given to the students right after they answered a question.From the interviews, it is clear that students were aware of the immediate appearance of feedback.All students interviewed expressed that they prefer immediate feedback.As for example, Student A2 prefers immediate.For her, receiving answers after answering all questions is difficult for her to remember and identify her mistake.Meanwhile Student A5 preferred immediate feedback because she can't wait to know the result.

Emotion
In line with their preference of the timeliness of feedback, the data in this study reveals that most of the students were having positive emotional impact after receiving immediate feedback through CBT.
Nine students were having positive emotion when receiving CBT feedback.As in the example, Student A1 felt excited when receiving CBT feedback immediately after he answered the question, no matter it is correct or wrong.He did not feel pressured if got wrong answer, but tried to find out his mistake by looking at the solution presented in the CBT feedback (Interview A1).
Similarly, Student A2, A4 and A5 shared that they felt "happy" when receiving the CBT feedback.In the particular, Student A2 expressed that she felt happy because she can know whether her answer was correct or not.Even though when her answer was wrong, this triggered her to look deeply on the feedback, tried to find out why she got it wrong: However, Student A3 expressed different emotion with other students.From the immediate feedback, he was able to know whether he got correct or wrong.However, for him, this created the feeling of tension in some degree.

Attention
Even though students perceived that the content of CBT feedback delivered was sufficient and important in detecting and correcting error, it is quite surprising that six students did not pay full attention on it.
As for example, Student A1 paid attention on the CBT feedback only when his answer was wrong.Besides, Student A2 admitted that she did not read through all the CBT feedback presented because of her "laziness".Other than that, Student A7 also expressed that he did not pay full attention on the CBT feedback.For him, he has no interest in reading all the sentences written in the feedback.He admitted honestly that he skipped the description in the feedback and only look at the example answer (working solution) provided.
Due to the poor students' attention towards the feedback presented, it is not surprising that most of them (nine out of ten) were not able recall and give example of the CBT feedback viewed.

Summary of Students' Perception towards CBT Feedback
From the qualitative data, students' perception towards CBT feedback presented through CBT in this study is summarized in Table 9.

DISCUSSION & CONCLUSION
Perception is important in understanding how a person thinks and gives meaning to something.Understanding how students perceive the feedback delivered through CBT is vital.What students perceive from their learning situation will influence their performance and also their motivation in learning (Annie, 2011).From the interview results, a richer picture of students' view of the computerized feedback was gained.Based on the results, students perceived feedback as explanation of correct answer, knowledge, and information.On the other hand, all students agreed that the content of feedback is sufficient.It is informative, accurate and relevant to the topic.Thus, it is useful in detecting errors made and correcting the error to improve their performance.This is in line with the findings by Randall and Zundel (2012) that students viewed feedback as useful as it indicated what they had done correctly and provide guidance to improve their performance.Moreover, the feedback presented is not only providing the correct answer (simple feedback), but also the verification that allow students to correct their error (answer feedback).As reported by Marsh (2012), answer feedback is necessary to maximize error correction.
Besides, students pointed out that they were able to identify their errors made from the feedback given.This shows that the feedback is useful in error detection.As supported by Mitrovic (2010), feedback plays the role of a mentor and informs the students of the errors made.Through feedback, students are able to judge their level of understanding and become aware of their errors made (Mason & Bruning, 2001).Similarly, the students also expressed that the feedback is useful in error correction.After reading the feedback, they were most likely not going to repeat the same error again.This is supported by Marsh (2012) that feedback is helpful in correcting students' errors.As pointed out by Gipps (1999, p.46), "learners require feedback in order to learn", students have the chance to learn from their error through feedback.It is an essential element in raising their awareness in relation to their errors which require further attention.In contrast, in the study by Kuiper and Pater-Sneep (2014), students expressed that they did not benefited from the feedback because what they had done wrong is not pointed out.Hence, this emphasizes the importance of the content of feedback given to students.It should be informative, especially able to point out students' weakness.Subsequently, help them to detect and correct their errors.
Despite the use of feedback in detecting and correcting errors, students admitted that there is inconsistency in error correction.Not all the students were able to correct their error in the subsequence similar question(s) nor they were able to correct their error absolutely afterward.This means that sometimes they answer the next question correctly after reading the feedback, but sometimes they get it wrong.For examples: The inconsistency in error correction might due to students' attention to the feedback provided.Although students viewed the feedback as useful, not all of them were paying full attention on it.Some students admitted that they only read the feedback when their answer is wrong, skipped through the feedback when their answer is correct.Besides, some of them even expressed that they looked at the correct answer (working solution) only rather than read through every sentences written in the feedback.This is in line with the study by Timmers and Veldkamp (2011) that attention paid to the feedback provided by a CBT varies greatly.In their study, fifty percent of the students paid attention to feedback of incorrect answers only.Meanwhile, twenty-five percent of the students did not pay attention to feedback at all.They suggested that the attention paid to feedback is influenced by task difficulty and test length.Despite emphasizing on the content and criteria of feedback, the findings of this study also reveal that there is a need to enhance students' attention on the feedback provided, especially their selfawareness of the importance of feedback have to be cultivated.Instead of trying the best to give feedbacks, teachers and educators should also help students to recognize the importance of feedback, be aware of the existence of feedback and how to utilize feedback in their learning.They have to know the importance of feedback and value the feedback received.Otherwise, it is a waste of designing and showing feedback to the students yet the outcome is not as fruitful as expected.
Besides, most of the students pointed out that they were having comprehensibility problem with the feedback presented.This reinforces the point made by Higgins (2000, p.1) that, "Many students are simply unable to understand feedback comments and interpret them correctly."Similarly, Carless (2006) found that some students perceived that they could interpret the feedback given but some found it is hard to decipher handwritten feedback or catch what the meaning in the feedback.Even though the feedback used in this study was proposed and designed carefully to ensure that it is clear and easy to understand, the findings reveals that there are two major factors affecting the comprehensibility of feedback among the students: (a) language and (b) sentences.It is in line with the study by Carless (2006) that even though the feedback was given on specifics, some students perceived that they could interpret the feedback given but some found it is hard to understand what the feedback meant.This triggers the idea about concerning students' language background and ability in giving feedback.
In terms of timeliness, all students prefer immediate feedback.They expressed that immediate feedback enables them to know the correct answer quickly, giving them chance to make improvement and build confidence.It is supported by Lilley, Barker, and Britton (2005) that students like to be assessed and get value comments on their achievement.Besides, Sambell, Sambell and Sexton (1999) suggest that lack of feedback can lead to student demotivation.Immediate feedback can provide remedial information constantly and avoiding confusion (Siegel & Misselt, 1984).Therefore, receiving feedback immediately motivates their passion in engaging actively in learning.
While the feedback presented is considered as positive feedback, most of the students are having positive feeling towards the appearance of feedback right after they answered a question.Feedbacks presented were written in positive component, such as "well done, keep it up!" and "good jobs".Negative statements such as "you are wrong" and "you are weak" are avoided.Affirmation instead of criticism was applied.As support that Hattie and Timperley (2007) and Lalor (2012) that affirmation must be given to students, emphasize their strengths and what's they need to do next to improve their learning.It should be descriptive rather than judgmental.This is in line with the study by Barrow, Mitrovic, Ohlsson, and Grimley (2008), which positive feedback provides assurance and reduces uncertainty to students.Hence, students are willing to receive the feedback presented, no matter their answer or correct as not.The findings of this study underline the significance of the role of positive component in a feedback.

IMPLICATIONS
This study pinpoints to advantage of the use of CBT in giving immediate feedback to students.Despite the importance of feedback, the frequency of feedback given to students in classroom is low due to the big teachers-tostudents ratio, teachers' limited time and many responsibilities/tasks in school.One of the key advantage of CBT is students can be assessed and receive valuable feedback on their performance immediately.From the findings, students preferred immediate feedback which delivered by CBT.They were motivated and able to detect and correct their errors.This may encourage teachers and educators to use it as a tool to provide detailed and instant feedback to students in a timely manner.

LIMITATIONS
There are some limitations in this study.Firstly, this study involves only Form One (Grade Seven in United States) secondary school students in Sabah, Malaysia.Besides, even though there are many subtopics in algebra, this study focuses on algebraic expressions, which is one of the topics in Form One Mathematics syllabus.Second, feedback is a powerful tool that provides information to students regarding their performance on certain tasks given.When feedback is given to students, certain criteria need to be met for it to be effective, especially promptness and timeliness.Therefore, computer-based testing was used as the medium of test in this study.For some students who are not computer literate, they might find difficulty in answering questions through computer.The effect of computer literacy on students' performance is beyond the scope of this study.

Figure 2 .
Figure 2. Example of Detailed Feedback Presented to the Students (Wrong Response)

Table 1 .
Respondents involved *A= StudentFigure 1. Example of Detailed Feedback Presented to the Students (Correct Response)

Table 3 .
Content of CBT Feedback Category

Table 4 .
Comprehensibility of CBT Feedback Category

Table 5 .
Usefulness of CBT Feedback Category

Number of Students Examples of selected statement
Selepas baca maklumbalas, boleh tahu… boleh tahu kenapa salah… ah… salah di mana (After reading the feedback, can know… can know… why wrong… ah… which part is wrong)" (Interview A5).] asked in Question 5… In Question 6, what has changed is the number only, working solution is still the same" (Interview A1)."Sebab… kalau soalan pertama salah, dia bagi maklumbalas, [jadi,] soalan seterusnya [saya] dapat jawab dengan senang (Because… if the first question is wrong, it gives feedback, [so, I] can answer the next question easily" (Interview A4). "

Table 7 .
Emotion towards CBT Feedback Category

Table 9 .
Summary of Students' Perception towards CBT Feedback