Asynchronous online collaboration as a flexible learning activity and an authentic assessment method in an undergraduate mathematics course

Online collaboration exercises were used as part of a diverse assessment package for an undergraduate differential equations course. Online collaboration served as a highly effective method for promoting and assessing generic graduate capabilities such as writing in a context-relevant manner and the development of self-awareness with regard to mathematical strengths and limitations. We present a number of examples of collaborations which can be broadly classified as "illustrative" or "corrective" in nature. The assessment strategy was found to be valid and largely reliable, although a number of issues arose with regard to reliability of peer-provided, formative feedback. These issues are addressed and suggestions for overcoming them are presented. Finally, a discussion of the successes of the strategy is presented along with a number of examples of positive outcomes resulting from the use of online collaboration as a learning activity.


INTRODUCTION
So-called "real world" applied mathematicians find themselves employed in a wide variety of occupational roles.From the traditional research scientist employed in academia or government scientific organisations (such as Australia"s CSIRO), to financial and investment risk analysts in banking and energy companies, and mathematical consultants employed by engineering firms, a common theme for a working mathematician is the ability to communicate mathematics.The problem with this is that not everyone is a mathematician -so not everyone understands what the mathematician does.
To become a successful real world mathematician it is essential to develop skills in communication in order to  understand problems of others,  translate a problem into mathematical language,  realise one"s own lack of understanding regarding a problem,  ask questions of a problem-poser, during the solution phase of a project,  communicate mathematical solutions in different (not necessarily mathematical) ways.The development of such skills requires university course designers or lecturers to provide students with authentic learning activities and assessment programs directly targeting these skills.
The research discussed herein has been undertaken at Queensland University of Technology (QUT).QUT is a large, inner city public university in Brisbane, Australia where all students reside off campus.For various reasons, including the traditional image of the university (carried over from its previous incarnation as its mathematical presentation, solution and written explanation.A similar strategy, appropriately redefined for the classroom, provides an authentic flexible learning activity and assessment method for undergraduate differential equations students, not to mention students of other subject areas.This strategy, and its associated results, is the focus of this paper. As Lawson (2002) notes, apparent advantages of online assessment systems and learning aids, such as constant and location-independent availability, need to be weighed objectively against any disadvantages.Such drawbacks include system-imposed changes to questions (when the computer system is unable to appropriately display mathematics problems or adequately handle responses), the inability to appropriately award partial credits and the apparent lack of higher level thinking required of many online systems.How such problems have been overcome in this course will be discussed later, along with a discussion of new problems which have arisen from this asynchronous collaboration approach to differential equations.
In the sections to follow the mathematics course and student group employed in this investigation presented in this study will be described.The assessment program, and in particular, online participation activities used in the course will be explained in detail.Finally, measures of the success of the online scheme will be discussed along with some problems which were observed and possible solutions for future use of the scheme.

Course and Students
The study described herein was conducted concurrently with the presentation of the course MAB413 Differential Equations (henceforth referred to as MAB413 or "the course").MAB413 is a second semester, second year undergraduate course offered by the School of Mathematical Sciences at Queensland University of Technology.This unit aims to provide students with a basis for understanding differential equations, their solutions and a number of solution strategies.The basic mathematical theory of differential equations, skills in the application of this theory, and the relevance of the material in this unit to real world problem solving are all developed.Among the generic graduate capabilities that are expected to be developed in the course are a) written communication appropriate to context, b) use of current technologies to advance the student"s learning, c) retrieval, evaluation and use of relevant information, d) development of awareness of the student"s own strengths and limitations, e) ability to work as a team on group work.These capabilities will again be mentioned when discussing the online collaboration which is the centre of this research.
The students taking this course are usually enrolled in the mathematics degree program or dual-degree programs comprised of mathematics and other areas of study such as engineering, business, science and education.A small number of physics degree students also enrol in MAB413.All students enrolled in the course are required to have completed studies in single variable calculus, elementary linear algebra, complex numbers and introductory differential equations (approximately two weeks in a first year course).It is also assumed that they have completed studies at the second year level in either multivariable and vector calculus or in further linear algebra, or both.
In terms of subject matter, the course is fairly customary, beginning with a revision session on first order initial value problems of linear, separable and homogeneous form, along with homogeneous second order, linear, constant coefficient equations.The students are then exposed to general linear differential equations of order n and the associated theory, before developing series solution methods especially for nth order equations with non-constant coefficients.Following this, the Laplace transform solution method is covered along with the related theory.The final two topics relate to systems of linear ordinary differential equations and then phase plane methods for nonlinear systems of differential equations.

Assessment Program
The assessment program for MAB413 was intentionally designed to be quite diverse in terms of the types of activities and the timetable of the activities.In a course such as this, which may be considered as a "solution techniques" course, rich in algorithmic methods that can be memorised and applied in examstyle situations it is essential to encourage students to learn more deeply and engage in the material at a higher level.This, according to Ramsden (2003), can be achieved via a diverse assessment strategy.
By engaging students and promoting deeper learning, the MAB413 experience will be more rewarding to students and provide them with skills, understanding and methods that can be used in studies in such areas as mathematical and statistical modelling, physics and further theoretical investigations of differential equations.
In particular, MAB413 included the use of four different types of assessment.The first being the regular participation activities, a part of which is the focus of this research, and will be discussed further in the next section.Briefly, this assessment item involved participation by the students from week to week in classroom activities as well as online learning activities, and was both formative in nature and summative counting 10% toward the students" final grades.A classroom test (midterm) was also held after the first 6 weeks of lectures were complete.This test served as both formative assessment, with detailed post-exam feedback offered to the students, and summative assessment since the test counted 20% to the students" final grades.
A significant, but optional project was also offered to the students which required well planned, semesterlong activity.The project was to be completed either alone or in a small group of up to four students.This item also served as both formative and summative assessment with regular feedback provided at "check points" throughout the semester, as well as optionally counting for up to 20% of a student"s final grade.If a student chose not to do the project, or to complete the project but found that they were not very successful, they were able to override the score with their final exam mark.
The final examination, a traditional end of semester style exam, allowed students to demonstrate the level of newly acquired knowledge in the material of the unit.It comprised a single 2.5 hour exam and required students to answer theoretical and practical questions on all sections of the course subject matter.This item counted for either 50% or 70% of the students" final grades, depending on whether or not they chose to complete and count their project grade.

Online Collaborative Participation Activities
The focus of this study is the use of online collaborative participation activities which formed part of the participation assessment activities briefly introduced earlier.While the participation assessment item also included classroom work such as collaborative tutorial activities and computer lab exercises, the activities of primary interest here are those conducted online.
Situational lessons discussed by Pandey (1990) for active engagement of students and preparation for problem solving in later life suggest the inclusion of different problem solving settings, the development of self-regulation, reflection and participation, as well as the development of thinking and reasoning skills.The online collaboration activities discussed here provide exactly this type of lesson.Students are provided with a collaborative environment online, not unlike the situation in modern workplaces.They are also given the chance to reflect on their own abilities and those of others while developing critical thinking skills and the ability to work with others for mutual advancement of knowledge in the subject area.
Under the system of definition of Roschelle and Teasley (1995), the activities discussed in this paper are more collaborative than cooperative in that they do not involve the division of labor in problem solving, but rather entail the mutual participation of students to solve problems together (also the Dillenbourg et al. (1995) definition of collaboration).

Summary of the Online Activities
In total, five participation activities formed part of the assessment for MAB413, however only 3 of these included online collaboration components.Over the course of the semester, the exact nature of the activity changed slightly.This was partly due to the time in the course in which the activity was completed (for example, near exams) and partly to accommodate suggestions from students and personal observations on the part of the lecturer.The activities are summarised below.
In Participation Activity 3, students were required to logon to a discussion forum on the course website and upload a worked solution to one of the questions in the course text from the chapter on Series Solution Methods.Students were required to type up their solution in either LaTeX or a word processor capable of equation typesetting (such as OpenOffice.org or MS Word).It was also allowed, but not preferred, for students to submit scanned, handwritten solutions.After some submissions had appeared, students were required to respond to two other students regarding whether or not they obtained the same answers and if they had not, why they believed they were right or where they had personally made a mistake.
Participation Activity 4 for Laplace Transform methods was quite similar to activity 3.However, students were directly instructed to use (partially) worked solutions for questions with which they felt they were having trouble.For both activities 3 and 4, students were summatively assessed in a very straight forward manner, according to whether they had uploaded a solution and whether they provided the appropriate responses to other students.This very simple assessment was used so as not to cloud the task with unwanted technicalities -the emphasis was on completing a worked solution and discussing solutions, suggestions and problems with other students.
The fifth Participation Activity was conducted toward the end of semester and close to the time of the final examination for the unit.The students were asked to construct a question which they believed was an exam-type question.It was suggested that the question should take another student about 20-30 minutes to completely answer and should not be overly easy or difficult.The section of the course on which the question was based was left open to the choice of the student thereby encouraging them to start thinking about a total revision of the course subject matter in preparation for the final examination.This time, the students were each required to respond to one other question, submitting a fully worked solution along with an evaluation of the question regarding its appropriateness as an exam-type problem.Following the submission of a worked solution, the original student was then required to respond to the student who submitted a solution.This response needed to say whether or not there was agreement regarding the submitted solution, to elucidate any mistakes made and provide suggested corrections.
By no means is this the only example in the literature of such a scheme.For example, Holliman and Scanlon (2006) discuss the use of near synchronous conferencing in a postgraduate distance education setting.The students, in small groups, engaged in the production of reports that demonstrated that both cooperative and collaborative learning were achieved in the class.This type of activity is similar to that considered in this paper, however the work of Holliman and Scanlon places less emphasis on the peer mentoring and teaching role which is quite prominent in the Participation Activities.While students are "mutually participating" to solve a problem, the "collaboration" of Dillenbourg et al. (1995), this is a different type of collaboration from that of Holliman and Scanlon which will be referred to as illustrative and corrective collaboration.
On the other hand, Aiken et al. (2005), discuss the use of an intelligent collaborative learning environment -in a sense, a virtual tutorial room complete with whiteboard, space for electronic documents and avenues for synchronous communication.The students in the Aiken et al. study are assessed on their collaboration and communication skills relating to a given task.The tasks are highly structured with the students required to employ "sentence openers" to initiate discussion.In the activities introduced in this paper, students simply offer mathematical solutions to given problems.The responding students were free to construct their own responses without prompts from such sentence openers as those used by Aiken et al. and this may have led to one downfall of the Participation Activities -that students sometimes show a lack of respect and/or understanding for the skill levels of fellow students.

Validity and Reliability
The summative assessment of students" responses was simple and clearly presented to students via a short rubric on the course website, prior to their undertaking the activity.With this type of previously available, commonly applied assessment rubric the summative portion of the assessment can be considered quite reliable with regard to accuracy and stability over the student cohort.The formative assessment however is a different matter with different students receiving different levels of feedback from the discussions.Both the course lecturer and most importantly, students themselves, were responsible for the formative feedback section of the assessment item and a number of inconsistencies arose throughout the course of the semester.For example, students who made the earliest submissions to the online discussion consistently received more feedback.This is only natural (a) since the student"s submission was available longer than those posted later, and (b) since other students naturally attempted to get part of the assessment dealt with as quickly as possible (the "response" part being seen as easier than the actual personal submission).
A further example of imbalance in the usefulness of feedback to students is that the higher-achieving students generally posted to the discussion first.Most feedback to higher-achieving students was from students at a similar level, while the students who struggled with differential equations posted later in the period and were less likely to make responses to early submissions and more likely to respond to other latesubmitting students.This can however also be seen in a positive light.Useful feedback was provided to students at both ends of the spectrum by other students at the same level and there was no apparent reason for concern for the instructor, that higher-achieving students were belittling lower-achieving students.
As outlined earlier, it was expected that outcomes of this course would include a number of generic graduate capabilities.These included communicating in writing in an appropriate manner; using current technologies to advance the student"s own learning; retrieving, evaluating and using relevant information; developing self-awareness of mathematical strengths and limitations; and being able to work as a team on group work.The online collaborative learning discussions which formed part of the participation activity for MAB413 provide an excellent means to assess these generic capabilities.Students were required to write both the mathematics of the question that they were answering, as well as the communication required to present their worked solution to others in the class.For example, some students presented worked solutions and then requested help from others in particular sections where they felt their understanding was lacking.Students also needed to develop and present written communication skills when they responded to others to provide their suggestions or corrections.
While the assessment of the "use of current technology" and "ability to work as a team" capabilities are reasonably obviously fulfilled in the online collaboration exercises, the "development of selfawareness" is touched in a more subtle way by the scheme.It seems on the surface that this is simply a peer-assessment exercise: a student submits a worked solution and then others comment on its accuracy.However, by requiring the students to complete both of these activities they have in fact become self-aware of there skills and limitations.This is first achieved as a requirement of the activity -students need to say whether they think another student is correct or not, and if not, why they believe that to be the case.In an unintended consequence of this assessment activity, students also showed self-awareness in the later activities (#4 and #5) where they actually offered up what they believed to be their own misunderstandings and requested others to help them.This, coupled with the constructive responses from other students, was seen as a major positive outcome of this online collaborative assessment strategy.
In terms of the generic capabilities listed in the course outline as expected outcomes of studying MAB413, the online collaborative participation activities appear to provide quite obvious and transparent measures of student achievement in each of the areas mentioned.As such this assessment item can also be considered valid in measuring that which it was intended to measure.

MEASURING SUCCESS
Measuring success of innovations in the classroom is often difficult and sometimes impossible to carry out objectively and to the satisfaction of independent readers.An obvious measure of the usefulness of this online collaboration activity would be to track the progress of students through their degrees and on to the eventual place of employment, along the way investigating the students" abilities to collaborate with colleagues and to use each other to increase skills and understandings.This however, of course, would require an unsupportable investment of time and money to have any hope of reaching any useful conclusion.Other possible measures of success can be obtained implicitly by observation of students" online collaborations, as well as through student evaluation responses and unsolicited comments.These measures will now be discussed.

Illustrative Collaboration
Useful collaborations between students in the online forum were observed on a number of occasions.By investigating, in their own time and at their own pace, the solutions of another student, it was possible for the students not only to uncover flaws in their logic or small errors in their working, but also to discover how their classmates interpreted directions of the lecturer or of the questions in the text.In Activity 3 for example, Student A submitted a worked solution to a problem requiring the power series solution of a differential equation.An excerpt of the discussion forum is shown in Figure 1.Numerous students responded by supporting Student A"s worked solution although these have been cut from Figure 1 to preserve space.Student B also supported the accuracy of A"s solution, but interestingly noted that he hadn"t considered incorporating any discussion of the theoretical existence of the solution (other than of course finding the solution) as was carried out by Student A.
The two students had different interpretations of the work required to "complete the question" and through the discussion forum this was illuminated and able to be dealt with quickly.
In a more obvious manner, Student C benefited from the forum model when she responded to A"s submission with "I almost got that but I made a mistake earlier on when substituting k, it was good looking at a worked solution!:)".This is a clear positive outcome of the forum where C has been able to locate an error in her work using the model solution provided by A. Naturally it may be argued that in-class or in-person collaboration would also have allowed for this discovery of error, however the online forum model allows for the  added elements of anonymity (if the student wishesnot taken in this instance) and of freedom to consider the worked solution in private.The student does not have their classmate with them at the time and so does not feel intimidated to agree with their working or to simply pretend that they have the same answers.This type of response was found quite often throughout all of the online participation activities in this course.

Corrective Collaboration
It was also observed that students submitting their solution would do so even when they were not correct or even confident of being correct.In fact, as was hoped when designing the activity, a number of students submitted incorrect or incomplete solutions with an aim to being corrected or helped by fellow students.Again in Activity 3, Student D submitted a worked solution, as shown in Figure 2, along with the message: "Hey.Ok I don't actually think this is right, but I cant [sic] figure out where I've gone wrong.Can anyone spot the error or give me a clue?Thanks.".Student E has then been able to consider the submitted solution and offer advice regarding a possible error near the end of the solution.The student has also offered some extra insight into the solution that they were able to glean from Student D"s solution.
The final post shown in Figure 2 demonstrates that following the reply of Student E, the original poster has been able to realize the mistake that they made in their original solution.

Student Evaluations of the Course
Toward the end of the semester students were able to complete a "Student Evaluation of Unit" (SEU) to evaluate and comment on the course they had undertaken.This evaluation instrument was made available online for a period of two weeks and students were asked to voluntarily submit responses to multiresponse questions as well as provide free-form comments on the course.All responses were kept anonymous by the evaluation system.
Two particular questions directly referred to the teaching methods used in the course -including the participation activities.11 of 13 students responding to the evaluation agreed (nine strongly agreed) that the teaching methods used in the unit worked together to help them learn, while two students reported to be neutral on the subject.All students agreed (nine strongly agreed) that the assessment tasks clearly related to what they were expected to learn in the course.While the response rate of the class was only 31% these responses are encouraging in that they reflect the appropriateness of the teaching methods and assessment tasks, at least in general.It indicates that in no way did the responding students disagree with the use of the participation activities.
The free-form comments were also a source of positive feedback regarding the participation activities.Responses to "Comment on aspects which are done well and should be continued" included: "The participation activities where questions are submitted and solved on the net!" "The participation activities were really good, especially since you had to upload stuff and got/gave feedback!!" "The participation activities work really well, especially the ones using the discussion forum, the [sic] should be continued." Regarding the unit and teaching in general one student commended learning strategies and innovations with regard to online learning.
Together, these responses and comments provide positive evidence for the usefulness of the participation activities both as learning activities and as an assessment method.

Problems, Solutions and General Trends
Throughout the semester, a number of general trends came to light which are interesting to note here for the interested reader who may wish to implement this teaching/assessment strategy.These are outlined below; along with a number of problems which were observed and possible solution strategies for these issues should they arise.
In the fourth participation activity where students were required to submit a worked solution to a question from the text book, a student from a non-English speaking background submitted a solution which, while mathematically correct, was somewhat lacking in the clarity of written communication.This in itself is not a problem since the student is not graded with regard to the written communication.
Unfortunately a responding student was not understanding of this fact, or the student"s background, when pointing out the original student"s lack of communication skills.This was in the no way the fault of either student -it should have been stressed in the classroom or online that students should be understanding of other students" backgrounds (for example, their original language) when making comments.It is also important to encourage a culture of understanding and mutual respect among the student group -as is always the case.Of course, there will always be situations where such an issue cannot be avoided and when this is the case it is important for the instructor to respond with a reassuring and measured response explaining the situation and how the students could improve their submissions.
Another problem which became apparent throughout the course is that students who submit responses later in the period for which the item is open will receive less feedback, and sometimes less useful feedback.More often than not, the better performing students in the class would submit and respond early while students who needed longer to understand concepts submitted later.
Two possibilities for overcoming this problem have been suggested by colleagues -one is for the instructor to respond to all submissions, thereby providing a (hopefully) "good" response to all students.The other involves the instructor splitting students into pairs who must respond to each other"s submission regardless of the timing of the submission.Holliman and Scanlon (2006) noted that tutor interventions were important in influencing peer interactions and in the activities considered in this research similar findings were made.It is important to note that this implies quite a substantial amount of time may be required of the tutor or instructor in order to produce quality learning outcomes from such online participation activities.
A positive outcome of the participation activity is that it provides a means of mass communication with the instructor, hopefully with less embarrassment for a student asking a question.For example, when students are required to submit solutions to a problem, and encouraged to submit solutions which they cannot complete, an outlet is provided for students who are otherwise embarrassed or uncomfortable asking questions.Since everyone must submit an incomplete problem, there should be less concern for the student to be concerned of other students" impressions.Furthermore, when the question is asked, either another student or the instructor provides an answer which is available to all students -not just the one asking the question.This is an obvious advantage over, for example, emailing the instructor.
Another trend which became evident over the course of the semester was that a number of students became aware of the need to explain and clearly present solutions.In a way, the requirement that students respond to classmates" submissions placed them in a pseudo-examiner role.The responding student was forced to understand that when an examiner grades a student"s solution, they must only look at what is written on the paper (or in this case, typed on screen) and does not know what is inside the head of student making the original submission.This is naturally a positive outcome when it comes to later in the semester when students are sitting for final exams, and hopefully later in life when students present work in real-life situations.
A further positive outcome of the participation activities is that students were able to observe alternative approaches.Quite often a responding student would make a comment such as "I got the same answer as X, but I did it this way".Such alternative approaches allow students to see easier, more direct methods, as well as more complicated methods which may be more general or include situations that may not have been considered in a simpler response.

CONCLUSIONS
In this study, online collaboration exercises were employed as both an assessment item and a flexible learning activity.The collaborative activities served well as an item to introduce and to assess graduate capabilities as well as to provide alternative learning strategies for students requiring a more flexible learning environment.
Examples of the collaborative activities have been presented in order to illustrate two different types of collaboration -illustrative and corrective -which are seen as important in the differential equations course.While a number of problems arose over the course of the semester, such as responses to non-native English speakers regarding written communication, there are also some obvious and not so obvious solutions which have been suggested for the interested reader.This teaching and assessment scheme is being used again in the author's current teaching allocation.This allocation involves the same differential equations course as well as a first year mathematics course with a far more diverse group of learners.A comparison of the usefulness of the scheme in these two different groups will be the subject of a future publication.

Figure 1 .
Figure 1.Participation activity 3 -a solution offered by Student A with responses from Students B and C discussing interpretation of question requirements and correction of a mistake, respectively.

Figure 2 .
Figure 2. Participation Activity 3 -a solution offered by Student D with a response from Student E and ensuing self response from D.