Exploring Students ’ Engagement Patterns in SPOC Forums and their Association with Course Performance

With the popularity of Small Private Online Courses (SPOCs) in higher education, a plentiful of discussion data has been increasingly generated in SPOC forums. With 752 undergraduates’ discussion posts, this study aims to investigate students’ engagement patterns within SPOC forums in terms of engagement behaviors and emotions. Firstly, we designed a behavioral code rule to identify postingand content-level behaviors, and examined their association with course performance. Secondly, we built an emotion lexicon including positivity, negativity and confusion word sets, and adopted an emotion calculation approach to visualize emotional evolutionary trends and to examine emotional differences in registration types and course performance. The results show that, (1) the high-performing group was more active in the most engagement behaviors except for interactive postings. (2) The registered group delivered more threads and wrote richer vocabulary in post content. (3) Whether students were registered for a course or not did not have a significant effect on their emotional expressions, but the registered group exhibited more confusion in forum interactions at the end of the semester. (4) Positive emotion was prevailing for the entire population. Furthermore, compared with the low-achieving group, the highperforming group had higher emotion densities in three types of emotions.


INTRODUCTION
In the past two decades, the tremendous development of information and communication technologies has promoted the innovation of teaching and learning modes in higher education.As a variant of Massive Open Online Course (MOOC), the Small Private Online Course (SPOC) is being universally used to serve on-campus students in formal education (Cheng, Liu, Sun, Liu, & Yang, 2017;Combéfis, Bibal, & Van, 2014;Liu, Cheng, Liu, & Sun, 2017).The SPOC mode has been verified effective for improving performance of students in engineering courses (Fox, 2013;Piccioni, Estler, & Meyer, 2014;Zhang, Zhu, Zou, Yan, Hao, & Liu, 2015).Unlike MOOCs, SPOCs facilitate the adoption of blended learning modes, which combines self-paced learning with supervised instructional strategies (Rooney, 2003;Ward & LaBranche, 2003).In blended learning modes, SPOCs can assist teachers to observe and guide learners as a supplement to face-to-face classrooms where individual learning data are difficult to be recorded (Cheng et al., 2017;Kaplan & Haenlein, 2016;Kim, 2007).

Student' Engagement in Course Forums
In SPOC-supported courses, discussion forums are commonly used as a crucial component in blended learning, and as the primary information exchange channel among students and their teachers.The increased use of SPOC forums has contributed to the generation of massive interactive data in higher education.Within the forum data, the hidden engagement patterns involving interactive behaviors and emotional states have been considered as critical factors indicating learning performance (Kent, Laslo, & Rafaeli, 2016;Phan, McNeil, & Robin, 2016;Qiu et al., 2016;Tucker, Pursel, & Divinsky, 2014;Yang, Kraut, & Rosé, 2016;Yang, Wen, Howley, Kraut, & Rose, 2015).Especially, interactive postings and average depth in threads were considered to be correlated with examination' grades (Kent et al., 2016).During discussion postings, students' positive emotion was connected to a good quiz performance (Tucker et al., 2014), and confusion and negative emotions had detrimental effects on their learning outcomes (Wen, Yang, & Rosé, 2014;Yang et al., 2015).Therefore, the engagement patterns in discussion forums are quite critical for identifying at-risk students in formal learning and improving their learning performance.Nevertheless, as for SPOC forums, the association between the latent engagement patterns and learning performance is still unclear and needs to be explored.

Research Purpose and Questions
As a reflection of student learning experience in discussions, a variety of discourse behaviors could be explored through forum posts, such as interactive postings between peers (Chiu & Hew, 2018), exchanges of learning feelings (Ramesh, Kumar, Foulds, & Getoor, 2015;Wen et al., 2014), expressing opinions (Ekahitanond, 2014;Liu, Zhang, Sun, Cheng, Peng, & Liu, 2016), etc.To get some insights of students' engagement in SPOC forums, we will explore students' engagement patterns in forum discussions in terms of two major factors: behavior and emotion.In MOOC contexts, the relationship between engagement behaviors and learning effects has been investigated in prior work (Wang, Yang, Wen, Koedinger, & Rosé, 2015), and the association between emotions and summative course completion has been assessed as well (Wen et al., 2014).But few studies have been conducted to unfold the engagement patterns in a restricted private course platform and connect them to final performance in degree courses.The aim of this work is to investigate the engagement patterns of college students in SPOC forums in terms of the behavioral patterns and emotional states, and their association with learning outcomes.Specifically, there are four questions to be addressed in this study: Question 1: What engagement behaviors in SPOC forums are associated with higher course performance?Question 2: What differences exist between engagement behaviors of registered students and non-registered students?
Question 3: What are evolutionary trends in emotional states of the registered and non-registered students, and what differences exist between them?
Question 4: What are evolutionary trends in emotional states of high-and low-performing students, and what differences exist between them?

LITERATURE REVIEW
Online discussion forums enable students to deliver feedback, discuss knowledge and skills, consult peers about questions emerged in learning, as well as facilitate students to form a sense of community that connects them to courses (Hollands & Tirthali, 2014).Recently, an extensive research focuses on engagement patterns within course forums and their relationship with learning effects.These concerns can be divided into two aspects, such as engagement behaviors and emotion states in interactions.

Contribution of this paper to the literature
• The purpose of this study aims to explore how students interact in SPOC forums and the engagement patterns behind different registered groups and different performing groups.
• This study indicates that, the registered group tends to be more active in delivering thematic posts than the non-registered group.There is not a significant emotional difference between registered and non-registered groups.However, the registered students had a higher level of confusion at the end of the semester.
• The study indicates that the high-performing group performed more actively in most engagement behavior categories except for interactive postings, and exhibited higher emotion densities across a semester.

Engagement Behaviors of Students in Course Forums
Engagement behavior is a critical indicator to identify the level of knowledge construction, affecting the ultimate academic achievement.Some existing research has implied a positive correlation between engagement in discussions and learning outcomes (Kent et al., 2016;Wei, Peng, & Chou, 2015).Especially, posting behaviors in MOOC forums have been considered a crucial predictive indicator to course completion or dropouts.Goldberg et al. (2015) explored that completers engaged in significantly more discussion posts than non-completers across all levels of education in a MOOC, but this study only investigated total number of postings instead of fine-grained engagement behaviors like asking questions, expressing opinions, etc.To obtain in-depth insight on engagement patterns, Tella & Isah (2011) designed a questionnaire to identify four prominent types of engagement behaviors in a university discussion forum, such as questioning, reaction posting, sharing of personal opinion and experiences and brainstorming.Cerezo et al. (2016) extracted some engagement variables such as active time spent in forums and problem-solving tasks, as well as number of words in posts to represent students' performance in course forums.In their study, the number of words in posts were considered as an index of the effort invested by students and found to be positively related with final marks.Anderson, Huttenlocher, Kleinberg, and Leskovec (2014) characterized students' engagement by connecting their assignment submissions to the discourse content within their postings, and they found that, the number of terms involving course terminology in forums was significantly correlated with the times of assignments' submissions.Liu et al. (2018) categorized the postings into interactive, registering, questioning, viewpoint, thematic and quoting behaviors.This study yielded that there were significant differences between high-and low-achieving groups in the former 5 behaviors, with the behavior most correlated to course performance being the questioning posting.Besides, by a content analysis approach based on cognitive science, Wang et al. (2015) identified three types of latent engagement behaviors (active, constructive and interactive) from forum conversations, in which active and constructive discussion behaviors were significantly related with students' learning gains, while interactive discussion behaviors could significantly predict learning gains only for inactive students in forums.
Unlike engaging with MOOCs, learners in SPOCs generally need to use their real identities to register for a course (Guo, 2017), and learners in a course may be quite familiar with each other since they typically come from one university or even a class with the same educational background.If engagement behaviors hidden in SPOC forums could be sufficiently extracted and investigated, the university administrators and course instructors could have more opportunities to reveal the latent reasons why some students perform well or not in blended learning.

Emotional States of Students in Course Forums
Separation of time and space in online learning contexts hinders the transmission of academic emotional information.In this case, it is difficult for instructors to identify the struggling, confused and even frustrated students.Previous research has indicated that positive emotion helps to promote students' learning interests and engagement levels (Altrabsheh, Cocea, & Fallahkhair, 2015;D'Mello et al., 2008), and students with an upsurge of emotion may have higher motivation to accomplish their learning goals.Ramesh, Goldwasser, Huang, Daumé III, & Getoor (2013) incorporated the positive/negative score of post content into engagement metrics to distinguish between disengaged and engaged learners in MOOC forums.They found that engaged learners had a higher performance score, but there no a significant difference between disengaged and engaged learners in terms of negative sentiment of post content.To explore emotional evidence behind MOOC dropouts, Wen et al. (2014) calculated the sentiment rate over time on posts and inferred the potential correlation between temporal sentiment rates (positivity/negativity scores) and student survivals.Chaplot, Rhim, and Kim (2015) incorporated the emotional scores of posts into a neural network to predict student attrition in MOOCs.The joint modelling of emotion and topics has drawn increasing attention on capturing latent emotional feedback and requirements during the processes of problem-solving (Liu et al., 2016;Ramesh, Dan, Huang, Daume, & Getoor, 2014;Ramesh et al., 2015).Besides, confusion has been viewed as an emotion to infer students' potential difficulties and final learning performance in online learning.The indicators of confusion in conversational cues have considered as a viable solution to detect students' confusion in interactive learning environments (Arguel, Lockyer, Lipp, Lodge, & Kennedy, 2017).Liu, Pataranutaporn, Ocumpaugh, and Baker (2013) claimed confusion might evolve into frustration and negative emotion when it could not be regulated appropriately.The students would be less likely to discuss course content if they were often exposed to confusion.Yang et al. (2015) utilized a collection of features (click behaviors, linguistic features and question features) to construct a classification model to automatically identify posts with confusion, and revealed that the more students showed their confusion in forums, the lower the probability of their continued participation in courses.
These studies have investigated plentiful regularities in MOOC forums, but quite few research focuses on automatically mining engagement patterns in SPOC forums.With increasing popularity of SPOCs in high education, the forum data needs to be sufficiently utilized to help instructors understand how students interact with peers in this type of restricted learning context and what engagement patterns are associated with course performance.

Participants
The data in this study is derived from two discussion forums of the Psychology Foundation degree course, which is a major course of science education within a primary and secondary teacher training programme.The forums are set in a SPOC platform issued by a Research and Development team at a Normal university of Central China (http://moocapp.starc.com.cn/application/mooc/).It is worth to note that SPOC forums are employed as a supplement to face-to-face instruction in regular teaching.
The collection and analysis of the data has obtained the permission of students and administrators in the university, and the personal identity information has been anonymized in this research.A total of 10,870 posts were delivered by 752 undergraduates across two semesters, such as the first and second semesters of academic year 2014-2015.The posters include 719 registered students in this degree course and 33 non-registered students who are only voluntary participants but do not need to gain credits of this course.Statistics on the two course forums are shown in Table 1.The average score of the registered students was 75.97 (on a scale of 0-100) with a variance of 13.87.Students' scores in the final exam were used to represent their course performance.The final exam aimed to evaluate a student's mastery of the psychological knowledge, skills and abilities to analyse actual problems.Thus, the evaluation did not consider students' engagement in the forum, which makes sense to investigate the potential association between perform engagement patterns and course performance.

Study Design
This study aims at exploring how students engage in discussions in SPOC forums with behavior analysis and emotion calculation approaches.In the first step, inspired by the study of Wang et al. (2015), we will formulate a behavioral code rule, and automatically identify engagement behaviors within each post by this coding rule.These behavioral variables include the explicit posting behaviors like launching threads, following a thread as well as the implicit behaviors like asking questions, expressing opinions.Furthermore, we count the content-based statistical variables such as number of sentences, notional words and average notional words per post for each participant.The association between behavioral variables and course performance will be analysed.In the second step, we will adopt an emotion density calculation criterion to quantify emotions within post content in terms of positivity, negativity and confusion, as well as explore temporal emotional trends of different performing groups and emotional differences between them.The detection and calculation of engagement behaviors and emotional states are implemented in the Java programming environment ECLIPSE 8.5, and the correlation analysis and difference examinations are conducted in MATLAB 2017's statistic toolbox.
It is worth to note, in order to divide the groups with different levels of course performance, the High-Low Discrimination index proposed by Kelley (1939) is adopted to determine the high-and low-performing individuals in terms of the top 27% and bottom 27% of final scores, respectively.In this way, we divide the students in each forum into the high-, medium-and low-performing groups.Then the corresponding groups of PF1 and PF2 forums are merged.Thus all registered participants (N=719, PF1: N=567, PF2: 152) in "Psychology Foundation" course forums are divided into the three groups, such as high-performing group (N=187, PF1: N=150, PF2: 37), the lowperforming group (N=190, PF1: N=152, PF2: 38), and the medium-performing group (N=342, PF1: N=265, PF2: 77) which will be not further analysed for simplicity.

Posting-level behaviors
Through the content analysis (De Wever, Schellens, Valcke, & Van Keer, 2006) on the forum data, we summarize five primary types of engagement behaviors covering 100% of posts.To ensure the accuracy of coding, taking randomly selected 50% of posts as baseline samples, we initially invite two coding experts from instructional technology to design the coding rule by manually identifying behaviors within the samples.Then we invite three researchers in learning analytics to jointly verify the coding results on one third of samples, which are randomly selected from baseline samples.After the three researchers reach a consensus on results, we compare corresponding behavioral codes with the results of experts.Finally, the comparative analysis unfolds that the kappa coefficient k reaches 0.95, indicating a high degree of inter-rater reliability on coding results (Landis & Koch, 1977).Thus, the coding rule is formulated as shown in Table 2.
In this way, the remaining 50% of posts will be identified with the coding rule above.Here a special situation may have occurred that a sentence could be identified by multiple behaviors.For instance, if a thread post contains an inquiry sentence, then it could be jointly annotated with LT and AQ.Therefore, a post-level behavior may contain some sentence-level behaviors.In this sense, some fine-grained engagement patterns could be detected within the textual context.

Content-level behaviors
The content within forum posts may reflect individual motivation and discourse behaviors according to the speech act theory of Austin (1975), which suggested that one person actually was conducting a certain behavioral tendency while writing words.Like the studies of Romero, López, Luna, and Ventura (2013) and Huang, Dasgupta, Ghosh, Manning, and Sanders (2014), we utilize numbers of notional words (NW) in all posts, notional words used per post (AN), sentences (ST), asking questions (AQ) and expressing opinions (EO) to respectively represent engagement indicators of each student in forums, wherein NW and AN are word-level features, ST, AQ and EO are sentence-level features.Especially, unlike the previous research, we only extract notional words referring to terms with specific notions, ideas or other actual meanings, such as a person, a thing, an act, an emotional orientation or an evaluation object, in contrast to a relational word without semantics.These words could be utilized to measure vocabulary richness of students in discussions.The three content-based variables would be measured as shown in Table 3.

Following a thread
Regularly follow a thread to engage in discussions, suggesting a follower only reply the initiator of the thread instead of other followers in a thread.

比如我经常读一些历史书籍，只是因为感兴趣，但
是我发现，在写作时会不时地用到书中的知识/For example, I often read some history books, just because of interests, but I found that I may occasionally use the knowledge in books in writing.

Interacting with peers
Mention or assist peers on a topic with keywords like "reply", "mention" or "@" in titles or content of posts.

DT Depth in a thread
Average number of followers under a contributed thematic post which implies attractiveness of topics launched and influence of a topic sponsor.

Registering personal information
Register personal attendances in forum with the requirement of instructor, keywords include "report", "I am", student' name and ID, etc.

Measure of Emotional States
In order to precisely capture emotional terms in posts, we adopt the Chinese emotion lexicon constructed in Yang, Liu, Liu, Min, and Meng (2014) to match terms indicating positive and negative moods or evaluations.Simultaneously, with suggestions of two psychology instructors on students' confused expressions, we formulate a confusion emotion word set with some Chinese terms indicating students' uncertain (e.g., "不确定/not sure", " 是不是/is or not"), hesitant or sceptical (e.g., "为什么/why", "真的吗/really"), incomprehensible (e.g., "不清楚/do not know"), inquiring (e.g., "请问/excuse me", "谁知道/who know", "怎么做/how to do") and other potential confused feelings in learning.The constructed lexicon and their distributions in discussion posts are described as Table 4.
To characterize emotion states of students, the concept of emotion density (ED) is introduced to represent the strength of each type of emotion over a week.Here a week is viewed as a sampling period to obtain a more significant emotional variation.It is worth to note that, unlike the research of Wen et al. (2014) focusing the sentiment ratio (positivity/negativity) in posts, ED reflects the level of emotional strength for each of positivity, negativity and confusion during a period, hence ED would be less than or equal to 1. Therefore, ED can be used to characterize varying trend of each emotion of a student during course progress.To this end, by the NLPIR Chinese word segmentation system (Zhang, 2018), we first split the content of each post into words by order.Then, the positivity, negativity and confusion words in the emotion lexicon are respectively picked to match valid emotional terms in the word sequence.The matching is a procedure of word-by-word scanning in order, and the negative /I", "认为/think", "心理学/psychology", "了解/know", "其他 人/others", "想/think", "选修/elect".

Notional words used per post
Indicate number of notional words averagely used for writing a post for a participant

AQ Asking questions
Ask a question to seek for answers.The specific indicators contain "?", "what", "why", "how", "which" or other key question symbols.
对于强迫症用厌恶疗法怎么治疗啊？/How to treat obsessive-compulsive disorder with aversion therapy?

EO Expressing opinions
Express personal views on a learning problem in subjective ways, including "I find", "I (do not) think", "I (do not) feel", or "I (do not) think", etc. within posts.
ED is calculated to indicate the occurring strength of an emotion category e over time t.For formulating the calculation criterion, we define the following symbols: •    : Emotion density of emotion category  ∈ {, , } during t-th week.
• (): Total number of sentences within all posts during t-th week.
To obtain a more consistent emotional information, we define the density of each emotion across the post set in a period based on the method of O'Connor, Balasubramanyan, Routledge, and Smith (2010).The calculation formula of ED can be described as follows: , ℎ(  ) > 0 0, ℎ(  ) = 0 (1) Here ED characterizes the evolutionary trends of collective emotions over course weeks by matching emotional terms on sentence level.Thus, the positive, negative and confused emotion densities (namely PED, NED and CED) can be calculated to detect the average sentiment strength of a certain group across a certain week.It is worth to note that the value of ED ranges from 0 to 1.The value of ED would equal 1 in an extreme case when all the words within   are emotional terms from the same emotion type.The value of ED would equal 0 when there are not any sentences or emotional terms during t-th week.

Engagement Behaviors and Course Performance
In this section, we adopt the designed coding rule to automatically annotate engagement behaviors within each post.Particularly, RP, DT, IP, FT and LT will be identified on posting-level since they are consistent with posting actions of students.As for the content within forum discussions, ST, AQ and EO will be identified on sentencelevel, while AN and NW will be counted to capture the words with actual meanings.

Association between engagement behaviors and course performance
To address the first question, all the behavioral counts are viewed as independent variables, the dependent variable is course performance indexed by each student' score in the final exam.The result of correlation analysis is shown in Figure 1.
The result reveals the degree of association of ten behavioral variables with course performance.First, there are significant correlations between most of behaviors and final grades.The frequencies of launching and following

Correlation coefficient
Behavior code thematic posts are significantly associated with students' final scores.Second, receiving more follows means that they might establish an attractive topic engaging peers to actively discuss within a thread.Also, the popularity of threads (indicated by thread depth) tends to be correlated with the publisher's course performance.Third, for the content-level behaviors, asking questions, expressing opinions, as well as writing multiple sentences and diverse vocabulary are all significantly correlated with course performance.This seems that, discussing with rich language, seeking for helps and sharing learning feedback with peers all indicate students' knowledge construction skills to some extent.
To explore the specific behavioral characteristics of the high-performing group, we examine the differences between the high-and low-performing groups with t-test on eight significant behaviors, i.e., LT, FT, IP, DT, AQ, EO, ST and NW, which exhibit moderate correlations coefficients with course performance.The t-test results on these engagement behaviors of the high-and low-performing groups are shown in Table 5.
The results above show that behavioral frequencies between the high-and low-performing groups are significantly different except in IP.The high-performing group tends to be more active than low-performing group in the seven behaviors of LT, FT, DT, AQ, EO, ST and NW.Relatively, LT (t(375)=23.70,p<0.001),FT (t(375)=25.41,p<0.001) and DT (t(375)=21.37,p<0.001) and NW (t(375)=15.69,p<0.001) seem to be most statistically significant in distinguishing the high-performing and low-performing groups.The possible reason is that, compared with lowperforming group, except for participating in more thematic postings, the high-performing group could offer relatively influential topics attracting more followers, and used richer notional words in discussions.In addition, both AQ and EO show significant differences at 0.05 level between the high-and low-performing groups, which implies the students in the high-performing group averagely proposed more questions and opinions than those in the low-performing group.The t-test results reveals that, the high-performing students have higher levels of engagement and influence than low-performing students in forums.However, on average, each student in both these groups only had fewer than two times of peer-interactions in writing discussion posts, suggesting these two groups seem to be similar and inactive in connecting peers during postings.

Differences between engagement behaviors of registered and non-registered students
For a restricted SPOC environment, the participation of non-registered students in discussions typically stems from the high-quality discussions and instructions in a course (Rothkrantz, 2016).Simultaneously, non-registered students need not to gain the course credits, and they engage in SPOC forums to pursue their own interests.It is worth to note that the 33 non-registered students in the Psychology Foundation course are completely spontaneous to engage in the course discussions.To address the second question, we conducted the difference examination between the non-registered and registered groups in terms of engagement behaviors.
As shown in Table 6, it could be found that the registered students performed more actively than the nonregistered students only on the three behaviors of LT (t(750)=2.15,p=0.043),ST (t(750)=4.68,p=0.003) and NW (t(750)=4.17,p=0.004).But these two groups have not shown significant differences on the other five behaviors.It seems that, with an excitation of academic achievement or requirements from instructors, the registered group launched more threads as well as used richer vocabulary and sentences in posts per week.This shows a higher motivation of the registered group in launching topics and expressing discourse content.On the contrary, the nonregistered group performed less actively in discussions.Seemingly, more external participants only observed the discussion processes as bystanders.Interestingly, the non-registered group received on average more follows after launching a thread.But the difference in DT was insignificant between these two groups, this is probably because there is a relatively large individual difference (S.D. = 7.95) within the non-registered group.

Emotional States and Course Performance
Emotion is a critical implicit factor affecting individual learning motives in online learning processes.Compared with conventional self-report methods (Broekens & Brinkman, 2013), the textual emotion calculation could automatically detect real emotional expressions of students online without any intrusive sensory device (Rodriguez, Ortigosa, & Carro, 2013).Unlike previous related research, we incorporate confusion as a type of emotion into the comparative study to perform a comprehensive emotion analysis combining positivity, negativity and confusion, which aims to investigate the relationship between academic emotions in forums and course performance.It is worth to note that, for each emotion, we sum up an overall evolutionary trend between the two semesters of Psychology Foundation course by averaging two sequences of emotion densities over weeks.

Differences between emotional states of registered and non-registered students
To address the third question, we respectively calculate PED, NED and CED for the registered and nonregistered groups across each week in a semester.The evolutionary trends of them are shown in Figure 2.For the registered group, PED, NED and CED show a similar evolutionary trend: there are two peaks during the first 7 weeks, they displayed a relatively plain trend during the middle term stage (8th to 9th week), and there is a peak in the 19th week at the end of the semester.Especially, for the registered group, each type of emotion typically had a sharp increase and achieved the highest level when approaching the end of the semester.In contrast, the nonregistered group shows larger fluctuations on PED, NED and CED across the semester.Interestingly, the students from the registered group tended to have a higher CED than non-registered group at last 6 weeks of the semester (17th to 22nd week), suggesting that the registered students expressed a higher level of confusion in the last few weeks of forum interactions.
Table 7 shows the paired sample t-tests of emotional states between registered and non-registered groups, which indicates that there are not significant differences on PED, NED and CED between these two groups in total.This indicates that, whether students were registered for the course had not a significant impact on their emotional expressions in course forums.In order to explore emotional differences within each group, we conducted the paired sample t-test for each emotion density between these two groups over these 22 courses weeks, as shown in Table 8.This seems that, within the registered group, there is a significant difference among PED, NED and CED, and the contrast relationship is that "PED>NED>CED".Within the non-registered group, PED still represented the dominant emotion state, which is significantly higher that both of NED and CED, but there is not a significant difference between NED and CED.

Differences in emotional states of different performing students
To address the fourth question, we characterize the evolutionary trends of PED, NED and CED for the highand low-performing groups over 22 course weeks, as shown in Figure 3.It can be observed that, the three types of emotions all show relatively high levels both at the beginning (1st-8th week) and the end (15st-22th week) phases of a semester, but show relatively low levels in the middle (9th-14th week) of a semester.In general, the highperforming group shows the highest emotional strength on PED, and achieves a peak level of PED at 19th week.In contrast, the low-performing group seems to have lower levels in all emotion densities.Especially, the highperforming group had a higher CED than the low-performing group over most time.The CED of low-performing students seem unstable during the whole semester, in which there are sharp increases at 3rd, 7th, 15th, 19th and 21st weeks.Specifically, the analyses on emotional differences are conducted within inter-and intra-group, respectively.
The emotional differences between different performing groups are shown in Table 9, there is a significant difference on each type of emotion density between high-performing group and low-performing group.The highperforming group tends to have higher emotional strengths than low-performing group on PED, NED and CED, among which PED can significantly distinguish from these two groups.These results reveal that the course performance is basically consistent with emotional strengths for these registered participants, and the highperforming students typically appear to have higher emotional arousals on PED, NED and CED.Moreover, an interesting phenomenon is that the high-performing group has a higher confusion than low-performing group.The possible reason is that the high-performing students tend to think about questions and express their confusion on course content to seek for help.This result is consistent with the comparative result on AQ between the high-and low-performing groups, suggesting that the high-performing group tends to be more active than the lowperforming group in asking questions.To examine emotional difference within each performing group, we first carry out an ANOVA analysis on three types of emotion densities for each performing group.As shown from Table 10, there is a significant difference among three types of emotion densities within the high-performing group (F(2, 717)=10.2871,p<0.001), while the difference among PED, NED and CED is insignificant within the low-performing group (F(2, 717)=2.7064,p>0.05).Through comparing the distributions of emotion densities of the two groups, we can find the students from the low-performing group tend to have less emotional expressions than those from the high-performing group.This may be related with the fact that low-performing students rarely engage in interactions involving opinion expressions, which results in a low level on each emotion.
In the post-hoc tests within each group, we find that the differences between PED, NED and CED within the high-performing group are more significant than those within low-performing group.For the high-performing group, the students tend to express more positive emotion in comparison to negative (t(375)=5.5805,p<10 -4 ) and confused emotions (t(375)=5.6310,p<10 -4 ).Relatively, for this group, the confused emotion shows the lowest level compared with the positive and negative emotions.For the low-performing group, positive emotional expressions significantly outperform negative emotional expressions, while the difference between negative and confused emotions is statistically insignificant.Actually, they often had both confused questions and negative feedbacks towards learning content at the same time.For instance, the post "我承认心理学与神经系统等生物学基础有着密不 可分的联系，但为什么第二章只谈神经系统而不说心理以及它们的联系，让人感觉晦涩难懂，产生了认知上的分离 /I admit that psychology is inextricably linked with nervous system and other biological basis, but why does the second chapter only focus on nervous system without psychology and the relationship of them, which makes me feel obscure, incomprehensible and have cognitive separation" written by a student in low-performing group implies that the knowledge about nervous system taught in Psychology Foundation course was both confusing and complex for this learner (both "why" and "obscure" indicate the confused emotion, "incomprehensible" indicates the negative emotion).Hence, the simultaneous appearance of the two types of emotions in a sentence may result in a small difference between them.

DISCUSSIONS AND CONCLUSIONS
In this study, we adopt the behavior analysis and emotion density calculation to jointly explore engagement patterns of students in SPOC forums across 22 weeks of observation.We examine whether students' engagement behaviors and emotional states are different in terms of their course performance or not.Thus, four research questions are addressed as follows.
Regarding the first question, we automatically extracted ten engagement behavioral features for each participant, in which launching threads, following threads, average depth of contributed threads are the three features most significantly associated with course performance.And the high-performing group was more active than low-performing group in discussions except for the interactive posting.Among these engagement behaviors, the deeper knowledge construction behaviors implied by interactive postings, asking questions and expressing opinions are lowly associated with course performance although the high-performing group performed more actively in these behaviors.This suggests that the behaviors associated with learning performance are mostly reflected in the quantity of interactions rather than the quality of engagement.Correspondingly, Shaw and Kathleen (2017) have shown that there was no significant relationship between forum quality and posting/replying behaviors, and these behaviors could not effectively explain forum quality, which is indicated by depth of discussion and critical thinking.Therefore, it is possible that the students were not prepared for higher quality discussions and there was a lack in effective guidance and stimulations of high-order thinking.The latter could potentially be supported via activities such as problems' creations and solutions (Tofade, Elsner, & Haines, 2013), situation-related topic discussions (Wang, Wen, & Rosé, 2016), feedback exchange among teachers and students (Ertmer et al., 2007), or assessment of discussion contributions (Klisc, McGill, & Hobbs, 2017), etc.
Regarding the second question, we conduct a comparative study between the registered group and the nonregistered group.The registered group was more active than the non-registered group in launching thematic posts.This result is consistent with the conclusion involving MOOC learners derived by Khalil, Kastl, and Ebner (2016), indicating that the voluntary external participants generally had a low posting frequency and certification rate compared with registered group.This phenomenon seems that a strong motivation to achieve good course performance had a positive effect on posting thematic content in forums.However, the registered students tend to be similar to the non-registered students in activity levels of most engagement behaviors.It seems that the course credit is not a major driving factor for students to engage in various activities (interacting with peers, questioning and expressing opinions) in forums.This result is opposite to that reported by Kursun (2016), which shows the credit bearing group had a significantly higher extrinsic goal orientation than non-credit-bearing group in engagement in MOOCs.This also implies that, the interactivity and incentives of course forums still need to be improved and diversified.If reasonably designed, SPOCs would attract different types of students (internal or external students) to equally engage in interactions in course forums.
Regarding the third question, we characterize the temporal emotional states and their differences between the registered-and non-registered students.Compared with the non-registered group, the registered group had a more and consistent evolutionary trend in PED, NED and CED across the whole semester.When it come to the end of the semester, all emotion densities tend to sharply increase, suggesting that the registered students had a higher emotional arousal in preparing exams.Especially for the confused emotion at the end of the semester, the registered group had a higher emotion density than the non-registered group.This phenomenon seems to be related to the learning objectives or motivation of students.For registered students, they faced the pressure of exams which are for obtaining an enough credit, while non-registered students did not need to face.Therefore, a registered student who needed to prepare for exams might have more confusion and thus they needed to request helps of peers, and the confused emotion typically occurred at the end of the semester.Interestingly, this is similar to the finding in Yang et al. (2015) which indicates that students' confusion was closely related to behavioral patterns in preparing online quizzes and exams.This seems that the exam might be a potential incentive to registered students' confusion.However, there are not significant differences between emotional states of these two groups in total.Only within each group, there is a significant emotional difference among PED, NED and CED, in which the positive emotion accounts for dominant proportion of all emotional states, suggesting an optimistic academic emotion among the entire population.
Regarding the fourth question, we detect the temporal emotional states of the high-and low-performing students.For both high-and low-performing groups, the temporal emotional states of PED, NED and CED all show a common trend: the high emotion densities have sharp increase at the beginning of the semester, then decrease at the middle of the semester, and finally have a sharp increase when approaching the end of the semester.This seems that the discussions at the starting and end stages both had incentive effects on emotional expressions.Compared with the low-performing group, the high-performing group had higher emotion densities over a semester.This indicates that the calculation of emotion densities might contribute to identify the potential at-risk students (average PED≤0.0050,NED≤0.0029;CED≤0.0022), which would be beneficial to improve the overall learning performance and success rate in SPOCs.Unlike our study, in the study of Ramesh et al. (2013), negative emotions did not show a significant discriminability on students' engagement/performance, which was considered to be related to the context in which the emotion was expressed.Therefore, the specific context (discussed topic, activity type, course content, etc.) in which each type of emotion was expressed might mediate the emotional difference between high-and low-engaging groups, and should be incorporated into the analysis of forum engagement in future work.As for the within-group emotion states, the high-performing group exhibits an emotional engagement pattern of "PED>NED>CED", i.e., the positivity and confusion were the most and least expressed emotions of this group, respectively.Nevertheless, within the low-performing group, there is a similar emotion density between negativity and confusion, which may result from the simultaneous expressions of these two types of emotions in a sentence.Interestingly, it can be revealed that the high-performing group seems to express more negative and confused learning feedback than the low-performing group, which is inconsistent with their performance.It remains inconclusive.One possible reason is that the high-performing individuals might more effectively solve the confused or difficult questions to digest some obscure knowledge, and thus succeed in final examination.Another reason may be that low-performing individuals were less involved in emotional interactions or exchange of feedback, thence their real confusion was rarely exposed into interactions, which is closely related with their low engagement in the forum discussions.
Thus, reports on engagement patterns could be valuable to establish a participatory SPOC learning environment for STEM students if they enable instructors to understand how students interact in forums, especially for those who lowly engaged in discussions as well as expressed negative or confused emotions.Based on these findings, instructors could adjust interactive teaching strategies and focus on the at-risk individuals with low emotion densities at a suitable time.Hence, these results may have critical implications for formulating targeted interventions to improve learning experiences and performance of students in technology-enabled formal learning environments (Arnold & Pistilli, 2012;Wise, 2014).

Figure 1 .
Figure 1.Pearson Correlation coefficient between students' engagement behaviors and course performance

Figure 2 .
Figure 2. Average emotion densities of registered and non-registered groups across course weeks (a) PED evolutionary trends (b) NED evolutionary trends (c) CED evolutionary trends

Figure 3 .
Figure 3. Average emotion densities of high-and low-performing groups across course weeks (a) PED evolutionary trends (b) NED evolutionary trends (c) CED evolutionary trends

Table 1 .
Statistics of the two SPOC forums in Psychology Foundation course

Table 2 .
Coding rule of posting-level behaviors

Table 3 .
Coding rule of content-level behaviors

Table 4 .
Distribution of emotional terms in dataset

Table 5 .
t-test results on eight significant behaviors of the high-and low-performing groups

Table 6 .
t-test results on eight significant behaviors of registered students and non-registered students

Table 7 .
Indicators of emotion states between registered and non-registered groups

Table 8 .
Indicators of emotion states within registered and non-registered groups

Table 9 .
Differences between high-and low-performing groups on each type of emotion density

Table 10 .
Distribution of three types of emotion densities and their difference on each performing group