This interactive workshop session invites participants with diverse technical and non-technical skills to work together to consider how data literacy impacts on learning analytics, both for practitioners and for end users. Learning analytics outcomes can be targeted at a wide range of end users, some of whom will be young students and many of whom are not data specialists.
This workshop will be held as a part of Learning Analytics and Knowledge conference 2016, University of Edinburgh, Edinburgh, UK, at April 26th 2016.
This workshop will encourage the sharing of knowledge and experience on this topic through a mixture of paper presentations and practical hands-on activities with datasets and visualisations.
As a starting point for afternoon practical activities and discussions, we will be providing access to the Open University Learning Analytics Dataset (OULAD) and other materials associated with the Open University Learning Analytics outputs.
We also encourage workshop participants, if they think it would be useful, to offer their own datasets, analyses, dashboards and visualisations to feed into the second half of the day (see data and other materials for details of how to contribute).
We invite contribution and participation from anyone involved in generating and communicating the outputs of data analysis, particularly to non-data experts as well as anyone with particular interest in developing visualisations or dashboards. We welcome non-data experts who want to contribute to a discussion on how to communicate around large data sets.
Whilst data literacy is rarely an end goal of learning analytics projects, this workshop aims to find where issues related to data literacy have impacted on project outcomes and where important insights have been gained.
Data analytics are coming out of the lab and into the mainstream. As more of the general population start interacting with complex data sets it is time to start taking data literacy seriously.
This is of particular importance within the field of learning analytics, where the results of data analysis are targeted towards a multitude of different end users, from young students, teachers, faculty and administration. Learning analytics aims to provide actionable insights to educators and learners on different aspects of learning. Whilst the remit of learning analytics can be quite broad, in each case there must be a communication of the data analysis through which the important insights can be easily understood by the target audience. Both the people producing the data analysis and the people on the receiving end must have a foundational level of data literacy in order to be able to communicate effectively via the data.
But how can this be defined? What are the specific competences of the different stakeholders and how, within the field of learning analytics, is it possible to support their acquisition or to develop an understanding such that it is possible to develop dashboards and visualisations of data that are easily understood by the target audience?
In early stages of the field of learning analytics a lot of focus has been on the types of data available and the types of analysis that can be applied to it. This workshop explores the issues around data literacy for learning analytics and asks the question do students and educators need a certain level of data literacy to understand the outputs of learning analytics? And crucially, do most students and educators achieve this level?
This workshop aims to highlight the need for a greater understanding of data literacy as a field of study, especially with regard to communicating around large, complex, data sets.
This section lists datasets and materials that will be used as part of afternoon activities. Note that it is not essential to have expertise in analysing data to participate in the workshop, nor is it essential for people wishing to analyse data on the day to look here in advance although they might find it helpful.
Please check back as links to additional data and materials may appear over time.
If you want to make your own data or other materials available in advance for any of the workshop sessions, please make them available from a web page and email a link to Jakub Kuzilek and we will publish the link here.
As a main source of data the Open University Learning Analytics dataset will be used. It contains data about courses, students and their interactions with Virtual Learning Environment (VLE) for seven selected courses. Data are stored in csv files. Dataset is certified by the Open Data Institute.
We invite both full papers and expressions of interest. Accepted long papers will be given a slot for presentation and will be published in the workshop proceedings. The expression of interest will be used to help us better understand the nature of workshop participants' interest in this topic.
Topics of interest include, but are not limited to:
For full papers please use CEUR Workshop Proceedings template.
*All submissions should be made through Easychair
In the morning we will have some short presentations based on submitted papers, but with a focus very much on speakers presenting information directly related to data literacy angles of projects.
In the afternoon participants will work in groups on one of three practical tasks. If necessary, there may be several groups working on the same type of task. Tasks are focused on understanding how data literacy can impact on communicating outputs of learning analytics, from different starting points. There will be flexibility on the day to allow for additional topics of interest to be explored. Proposed tasks are:
Participants are invited to look at available datasets (see Data and Other Materials) in advance of the workshop, to think about analyses they would perform on this data, and who would be the intended audience.
During the workshop, participants will work together to analyse and produce insight from a more extensive version of this dataset, which they will then try to communicate to other workshop participants who will act in the role of different stakeholders.
Participants are encouraged to bring their own devices and preferred tools for data analysis and visualization.
This group could contain a mixture of participants who are proficient in data analysis and visualization, as well as non data experts.
The types of insights we might expect from this group would be a developing understanding of how choices of analysis impact on the types of visualisations and communications that are available from data. Data analysis experts may be prompted to reflect more carefully on how they choose to communicate the outcome of their analyses with respect to the assumed competences of their target audience.
Participants in this group will be provided with a pre-digested output from the Open Universities (OU) own analytics work on a) predicting learner failure from interactions in the VLE b) providing student focused recommendations.
The output will be derived from the OU's existing visualisations of data analyses, yet will be presented in a non visual way.
Participants are encouraged to bring their own examples of outputs from data analyses that have yet to be presented to a user.
Participants will be asked to identify ways to communicate this data to different types of user, including expert and non experts, or faculty, tutors, or students. They will present their outputs to other groups to assess how successful they are in conveying the intended message.
Outputs could be created by technology or drawn onto paper as a creative design, or constructed in 3D. Participants are invited to be particularly creative in this session. The types of insights we might expect from this group would be around how to communicate to different audiences from complex datasets and the creation of potentially new modes of visualizing learner data. We hope to create a better understanding of how to visualize and communicate to students themselves the outcome of recommenders.
In this group, participants share their experience of creating dashboards and visualisations for different types of users.
Participants are encouraged to bring examples of visualisations they have used with end users as a point for discussion, to understand if the outputs are communicating effectively and to collectively explore better ways for visualizing, especially for non-expert audiences.
The types of insight expected from this group are towards developing best practice for communicating about data, based on real life examples.
Groups will share their insights amongst each other at the end of the day, insights will be captured and used to define more clearly what data literacy means in the field of learning analytics and what are the biggest problems to tackle.
The workshop day is 26th April and will run from 8.30 until 16.30. Presenters have a twenty minute slot and can decide for themselves how to use that time to present/discuss their paper. The agenda is as follows:
| Time | Event |
|---|---|
| 8.30-9.20 | Introduction and welcome |
| 9.20-10.00 | Paper session 1 – MOOC |
| The Role of Data Literacy within a MOOC Analysis | |
| Learning Dashboard: Bringing Student Background and Performance Online | |
| 10.00-10.30 | Morning Tea |
| 10.30-11.00 | Paper session – classroom |
| Data Literacy in the Smart University approach | |
| Analysing performance of first year engineering students | |
| 11.10-11.30 | Introduction to datasets (OULAD and MOOC data) |
| 11.00-12.00 | Organisation of groups and beginning tasks |
| 12.00-13.00 | Lunch |
| 13.00-14.30 | Practical work in small groups |
| 14.30-15.00 | Afternoon Tea |
| 15.00-15.45 | Continuation of group work |
| 15.45-16.30 | Report back from group work and discussion of next steps |
| 16.30 | Close |
Martin Hlosta is a Research Assistant at Knowledge Media Institute at The Open University and PhD student at Faculty of Information Technology, Brno University of Technology, where he received his master's degree in Computer Science in 2010. His current research interests include data mining methods in learning analytics and learning from imbalanced data.
Jakub Kuzilek is Research Associate at the Open University, UK and also Research Assistant at the Czech Technical University, CZ. He is the member of OU Analyse project and his professional interests include machine learning, signal processing and learning analytics.
Dr. John Moore is a Research Engineer in the faculty of Maths, Computing and Technology at the Open University. He is currently working on the MK:Smart project where he is focusing on Electric Vehicle (EV) mobility. Part of this project involves carrying out data analytics on large quantities of real-time data. He has spent several years in HE as a senior lecturer in computing and has taught across all levels. His research is focused within the Internet of Things (IoT).
Annika Wolff is a Research Fellow in the faculty of Maths, Computing and Technology at the Open University, whose interests lie at the intersection between big data, machine and human learning. She is currently working on the MK:Smart project, using complex urban data sets for teaching data literacy in schools. Other research topics include learning analytics and data visualisations for tutors and learners and the use of games and narratives to motivate learning. She has worked on a number of UK and EU funded projects.
Prof. Zdenek Zdrahal is Professor in of Knowledge Engineering at the Knowledge Media Institute of the Open University, UK. He leads the OU Analyse project, which aims at the development of methods and tools for learning analytics and their applications at the Open University, UK. His research interests include the application of AI in design, case based reasoning, information extraction, predictive modelling, machine learning and knowledge sharing.
