Learning Analytics and Learner Activity

While data from a Learning Management System (LMS) can tell you about assessment scores and percentage completions, it can also tell you a lot about your learners, the content and their interplay. To get performance insights of your platform and the learning content, you must identify different data points, fetch the data, analyze and interpret it. Learner proficiency, Learner engagement, Learner preferences and Learner progress are some metrics that are computed by combining several data. Some examples of data that you collect from a learning platform are Clicks, Pages visited, Time spent, Search keywords, etc. There are some metrics that are directly connected to learner activity and we will look at those in this post.

Learner activity

You can gauge how active learners are by measuring and studying the frequency of certain events on the platform such as the number of times a learner logs in to the platform. Learner activity gives you a direct measure of how busy the platform is and can help you answer some questions about your learner’s motivation and interest levels. It may also give you indications about some learner preferences.

Types of learner activity

The regular activities of learners on a platform obviously depends on what the platform allows learners to do. If the platform is a basic LMS, then the activity of a learner would be viewing modules and taking assessments. However, if your platform has powerful features then a learner would also explore new content, interact with other learners (social learning), participate in contests, create content on their own, answer surveys, etc. and also navigate between all of these features. All learner activity is checked for a given period or a recurring periodicity.

Chart showing number of times modules were accessed during a week

Due to the nature of this measurement, i.e., frequency of activity over time, I’d like to think of these learner activity measures as ticker-timer metrics. In other words, if you simply plot each occurrence of the activity on a ticker tape, it can help you visualize the trends.

Observation: Learner activity was high in the morning

Observation: Learner activity was high in the evening

Observation: Learner activity was uniform throughout the day

Observation: Learner activity was uniform throughout the day but less frequent

Observation: Learner activity was random

Periodicity for measuring learner activity

Data analysis is done for a given periodicity. Below are some periodicities and some questions that you should ask:

• Daily: This periodicity tells you about your learners' daily patterns and can give insight into whether they learn during working hours, before or after. If your learners are more likely to be active after 7pm, the notifications and content must be pushed to them between 5pm and 6pm and not during the mornings when they are likely to pay no attention to it. Here are some questions you can ask:
  • When are my learners most active?
  • What time should the content and notifications go to learners?
  • Are learners not finding enough time during working hours to learn?
  • Are learners expected to complete learning beyond working hours?
  • Is the minimum learning seat time so high that it affects work?
  • Are daily learning patterns affected by other parameters such as nature of work or gender?
• Weekly: Weekly learning patterns will tell you if learners prefer completing their activities on the weekends or during the week. Here are some questions you can ask:
  • Which are the busiest learning days of the week?
  • Do users spend time on the weekend to learn?
  • What is the ratio of weekend learners to weekday learners?
  • Are learners only busy after a module is published to them?
• Monthly: Here are some questions that you can ask when studying monthly patterns:
  • Are learners busy only in the last week of the month? (perhaps, an indication that the learning is done only for compliance and learners are actually not motivated)
  • To what extent does gamification such as contests and redeemable points affect monthly learning? (is learning becoming seasonal)
• Yearly: Here are some questions you can ask when studying yearly learning patterns:
  • Are quarterly, half-yearly and yearly targets and closures affecting learning?
  • Are holidays affecting learning? (maybe you’ll want to then keep learning out of the holiday season)
  • Are learners completing courses just before appraisals? (again, a compliance aspect when an employee is required to complete a minimum number of learning hours every year)

Measuring Learner Activity

Basic activity can be tracked by checking all access points such as logins / logouts, modules accessed / closed, assessments accessed / closed, surveys started / ended, etc and can be collected for different cohorts (departments, position, region, gender, etc) and compared. Here are some things you could look at:

• Frequency (How often): If learners come in very frequently to the platform, it could mean that they are taking up multiple courses in parts throughout the day. It could also mean that the platform or the course has usability or accessibility issues.
• Time Spent (How long): This measure is derived from the in and out times of an activity. How much time does a learner spend learning? If people are spending more time on the platform it could mean that the platform is exciting or it could mean that the platform is slow or the content is long.
• Time of Occurrence (When): When do learners become active – during the day, during the week, during the month and through the year? Learners visiting frequently just before a learning completion deadline may mean that they are not motivated to learn but are only meeting L&D targets.

Interpretation of data and final thoughts…

Fetching data is one thing and interpreting it is another. You may have to combine multiple data to correctly interpret a behavior. For instance, search keywords used by learners can tell you what modules your learners are looking for. It does not however tell you if it is because the topic is trending in the industry or is it because the modules assigned to them on that topic are not good enough. As a final note, you must remember to share the findings related to content with Instructional Designers and Content developers. They can then use the information to make better choices when building courses that will give learners a high level of engagement. So, what’s the learner activity trend on your LMS?