Search

Each year since 2004, a new Horizon Report has been released. Each edition attempts to forecast the most promising technologies likely to impact on education along three horizons: the short term (the year of the report), the mid-term (the next 2 years) and the long term (the next 4 years). This paper analyzes the evolution of technology trends from 2004 to 2014 that correspond to the long-term predictions of the most recent Horizon Report. The study analyzes through bibliometric analysis which technologies were successful and became a regular part of education systems, which ones failed to have the predicted impact and why, and the shape of technology flows in recent years. The study also shows how the evolution and maturity of some technologies allowed the revival of expectations for others. The analysis here, which focuses on educational applications, offers guidelines that may be helpful to those seeking to invest in new research areas. [Copyright &y& Elsevier]

Peer assessment activities might be one of the few personalized assessment alternatives to the implementation of auto-graded activities at scale in Massive Open Online Course (MOOC) environments. However, teacher's motivation to implement peer assessment activities in their courses might go beyond the most straightforward goal (i.e., assessment), as peer assessment activities also have other side benefits, such as showing evidence and enhancing the critical thinking, comprehension or writing capabilities of students. However, one of the main drawbacks of implementing peer review activities, especially when the scoring is meant to be used as part of the summative assessment, is that it adds a high degree of uncertainty to the grades. Motivated by this issue, this paper analyses the reliability of all the peer assessment activities performed as part of the MOOC platform of the Spanish University for Distance Education (UNED) UNED-COMA. The following study has analyzed 63 peer assessment activities from the different courses in the platform, and includes a total of 27,745 validated tasks and 93,334 peer reviews. Based on the Krippendorff's alpha statistic, which measures the agreement reached between the reviewers, the results obtained clearly point out the low reliability, and therefore, the low validity of this dataset of peer reviews. We did not find that factors such as the topic of the course, number of raters or number of criteria to be evaluated had a significant effect on reliability. We compare our results with other studies, discuss about the potential implications of this low reliability for summative assessment, and provide some recommendations to maximize the benefit of implementing peer activities in online courses. • Based on Krippendorff's alpha, we have analyzed the reliability of peer assessment in MOOCs. • The dataset includes 63 activities, 27,745 validated tasks and 93,334 reviews. • The results obtained clearly point out the low reliability, thus low validity. • There is not relationship between the topic of the course and the reliability. • Courses with more than six tasks present an improvement in the reliability. [ABSTRACT FROM AUTHOR]