Classification Approaches in Learning Analytics, Does it always give the better results ????

In educational data mining many researchers have tried many different approaches available in data mining context to predict the learning patterns of the students to achieve better and quality results. All these approaches are mainly focusing on getting the results according to a particular student domain which highlights various specific features indicated through the Learning Management Systems.

Virtual learning is growing enormously and the student population those connect with these Learning management systems are increasing by numbers every day. With the necessity of understanding of each student learning pattern teachers should have a better way of predicting the performance of their students. In response to this necessity different classification techniques can be used to compare and interpret the educational data and improve the modeling of students in to different categories.

In order to make the student modeling process much easier Diego Garcia Saiz and Marta Zorrilla [2011] have researched on applying different classification techniques on the student data to predict their performances. In their research they tried to implement a tool known as Elearning Web Miner (EIWM) to discovering how the students are behaving and progress in the courses which is very helpful for the tutors to identify the students who need more attention among from a larger set of students.

One of the main reason that applying learning analytics in educational data sources is challengeable because of the dataset becomes very small comparing to the other application we see around us. Even though the number of student information which contains in a database is huge, most of these are dynamic and contain many variations among them. Since for this research they found it difficult to collect required data which made them to use the data for past three academic years for average student enrollment of 70  per year for a specific course module. For all of these student instances they considered attributes with mean values such as total time spent, number of sessions carried out, number of sessions per week, average time spent per week and average time per session.   

With the intention of analyzing and choosing  best classification algorithms for educational datasets they  analyzed four of the most common machine learning techniques, namely Rule-based algorithms, Decision Trees, Bayesian classifiers and Instance-based learner classifiers which are mainly were OneR, J48, Naive Bayes, BayesNet TAN and NNge. 

They tested these five algorithms using different parameter settings and different numbers of folds for cross validation, in order to discover whether they have a great effect on the result. In the evaluation process they found that Bayes algorithms perform better in accuracy and is comparable to J48 algorithm although it is worse at predicting than Naive Bayes which is the best in this aspect. Due to the results they achieved they highlighted that OneR suffered from over-fitting in this dataset, so that it should be discarded as a suitable classifier for very small datasets.

They also observed that NNge improves its performance in this dataset although the great number of rules which it offers as output makes it less interpretative for instructors than the rest of the models. Finally they conclude that Bayes Networks are suitable for small datasets in performing better than the Naive Bayes when the sample is smaller. As consequence of the fact that BayesNet TAN model is more difficult to interpret for a non-expert users and J48 is similar in accuracy to it.

One significant result which I see in their research is that the pre-processing step which they followed. In the dataset they found that there are instances which can be considered as outliers in the statistical sense and they suggested a mechanism to remove or eliminate those outliers in the data set which can improve the results by 20%. This makes a huge advantage when the data set is larger in size and provide with better quality results for the users.

What I believe about this research is that even though they suggested these approaches in classify the students, it cannot be proved that the same algorithm is suit for every situation we have in the educational domains. Some algorithms can perform well with small datasets and some can perform well with larger data samples and some are providing more interpretable results and some are not. Therefore depending on the problem situation and the context we have to choose the best algorithm that can be used for the specific process so that we get more acceptable quality output as final results.

Reference: D. García-Saiz and M. Zorrilla, “Comparing classication methods for predicting distance students’ performance,” 2011.