Why do we need early warning signals?
Some students encounter problems at the start of their studies but often this becomes visible when the exam (at the end of the course) is not passed. By signaling suboptimal study behavior in the first weeks of a course study and providing timely interventions behavior can be changed. Interventions can be diverse such as advice from a study advisor, extra guidance from a lecturer, or students studying more or differently. Analytics for Learn (A4L) can provide early warning signals. The use of signals in higher education in the Netherlands is relatively new. Research in the VS (Purdue University) has shown a strong relationships between stent behavior in the electronic learning environment and study success. The EWS project is part of the large project Progress Volg.
What is Analytics for Learn?
Analytics for Learn is a system from Blackboard that provides course analytics in a report format. Information about activity and grades in the electronic learning environment (Nestor/Blackboard) will be combined with information from the Student Information System (SIS/Progress.NET). A4L transforms this information into insightful reports for lecturers, students and study advisors. Reports for lecturers and students are accesable in the course environment, reports for study advisors are accessable from a RUG analytics environment. Reports can show the activity and/or grades of a single student compared to the average of all other students in the course. It is important that teaching assistants, study advisors or other lecturers have the 'instructor of assistant' status. If they are enrolled as students their actions will be incorporated in the reports.
Why is Analytics for Learn interesting for me?
Analytics for Learn grants you the ability to analyze the behavior and performance of your students in closer detail and directly from the start of a course. In this way you can identify students who might need support.
Why is Analytics for Learn interesting for my students?
Students can access a report in their course environment which shows them their own activity and grades compared to the average of the other students in the same course. Students can reflect on their activity and performance and increase their efforts if necessary or seek support.
The EWS project is explained to students in more detail during a 2 minute movie which is available on YouTube via: https://www.youtube.com/watch?v=j0ZkDiSDtGo
Which reports are available in the A4L Reporting Services Portal?
There are about 20 reports to be accessed in the Reporting Services Portal. However, not all of them are presenting the correct results. This portal is designed for the United States education system which is not 1-to-1 translatable to the data systems the University of Groningen uses. For example, because some courses are linked to more than one program and programs have sometimes more than one study advisor, we can not link a student to a study advisor. Therefore it is not possible to use the 'Personal tutor reports' at this point.
If you use the "back" button of your browser in the reports you will have to select everything anew again. It is therefore recommended to open any links in a new tab instead so that the selections are not lost.
During the pilot phase lecturers will have access to the reports in the reporting services.
The following 5 reports will be tested in the pilot phase:
1/ Activity Matrix: shows the relationship between activity and performance in four quadrants for each student in a particular course
2/ Grade Center Exception Report: shows an overview of grades of each student in a particular course
3/ Learn Course at a Glance: shows an overview of the level of activity for each student in a particular course
4/ Log-in Exception Report: shows an overview of the log-in details for each student in a particular course
5/ Student at a Glance: shows the activity of a particular student in a particular course.
Important: the Reporting Services underlying data structure is structured in a way that you always have to choose a course or a student. At this moment it is not yet possible to choose a program (opleiding), cohort or degree (fase; propedeuse, bachelor of master).
How can I access the reports?
The Analytics for Learn Reports can be accessed at http://a4l.citesi.nl. Here you can login with your username and password. This is not your usual username/password combination, but the one created specifically for this purpose and handed out to you during the pilot introduction. Remember to use the prefix bbpr\ in front of your username, as shown in the screenshot below. If you don't remember your password then visit this link to manage your Analytics for Learn account details. For some tips on using the reporting portal, please see this article.
Now you can press the button "Analytics for Learn Reports" to go an overview of all the reports you can use.
You will be presented with an overview of all the reports that are available to you. Highlighted are 5 reports which will be tested in the pilot phase.
The use of 3 A4L Reporting Services Reports cases
Below we have described three use cases in which three A4L reports from the RSR portal can be used. You can click on the use case to see a detailed description of how this works.
- Learn Course at a Glance: Examine the level of activity of students in a course in order to see which students are less active in the Nestor course
- Student At A Glance: Taking a detailed look at the activities in the Nestor course
- Activity Matrix: Examine the relationship between activity and performance
A note of caution
The aim of the A4L reports witing the context of the Early Warning Signal project is to provide as early as possible signals that might reflect an increased risk for study-related problems.
Please bear in mind when interpreting the reports or discussing them with a student or colleague that the report shows no hard evidence of student's efforts nor time spend studying. If all course materials are available at the start of a course, a student can download all content in the first week. He/she can study this content for hours (outside of Nestor) or not even look at the material at all. Not downloading the material doesn't mean the student has not studied the content because he/she might have received a printed version from a fellow student. In addition, a student who seems inactive in the digital learning environment might follow all classes, makes extensive notes and is actively involved in discussions within and outside class.
Questions and support
For technical questions, please contact Nestorsupport at firstname.lastname@example.org or call us at 050 363 8282.
If you don't remember your password then visit this link to manage your Analytics for Learn account details.
If you want to know more about Analytics for Learn or the Early Warning Signals project or if you want to take part in the pilot, please Esther Bouma (email@example.com or 050-3636434).