Data driven organisations

What does this mean for education?

Analytics profoundly shape the educational reality that they measure. What is measured and reported through the use of infographics or dashboards becomes more important than what is not reported. All levels of education are becoming data driven organisations. ‘Big Data’ and the use of analytics can provide insights into some of the gnarly challenges associated with improving equity and excellence. Governments are using it and businesses are selling it. For example:

• OECD uses PISA data (mainly achievement data associated with conditions of learning) to tell the world which country has the best education system.
• Individual countries use achievement data to show which schools have high and low levels of achievement.
• Online learning tools can be bought to provide moment-by-moment information for students, teachers and school leaders about individual student’s performance.

And in each case, the expectation is that the analytics will guide next steps. There is an expected ‘now what?’ response. Governments are guided to improve in their education system, schools are guided to know which students need targeted support and teachers are guided to know what the focus of support should be.

The key thing is human assumptions underpin data collecting, analysing, interpreting and reporting and these assumptions are then applied to the tools and analytics. For example in the national and international analytics it is assumed that literacy, mathematics and science achievement are essential life skills and signal that a country is preparing young people for the future. One problem with this is that readers of the reports may ‘forget’ things such as literacy is of service to the curriculum (and is not the curriculum). For example, in New Zealand student success is about students being “confident, connected, actively involved, lifelong learner” (and achievement both leads to and is becuase of student success) (NZC and expanded in ERO indicators).

For learning organisations to be data driven organisations, all assumptions should be transparent and checked that they align with the purpose of education and the outcomes we want for our young people:

• Who are the ‘we’? (does it include the young person, their family and their teachers?)
• How can we ‘tell’? (are we relying on what appears easy to measure or are we gathering a deep picture; and with what confidence does this data tell us?)
• What information will be used in the ‘digital profile’ (does it reflect the whole person; and what information will not be used?)
• What counts as learning and achieving (does it reflect our vision for young people and the experiences we wish them to have; and how is ‘what counts’ decided?)

The assumptions brought to these questions are embedded worldviews, and beliefs about young people, education, pedagogy and curriculum. We need to ask ourselves

“Do the analytics we use support all learners to develop the knowledge and capabilities needed to be confident, connected, actively involved, life-long learners, and all educators to provide young people with holistic educational experiences or are we using them in ways that will perpetuate inequality and will not realise potential?”.

If the purpose of analytics is to provide feedback as to whether the intent of the teacher/ leader/ implementers/policy makers’ actions is the reality in learner outcomes then analytics can change education in at least three positive ways.

1. Analytics can provide faster feedback loops for students as they can provide rapid adaptation of teaching actions and therefore quicker learning for students. In the past research knowledge was not easily accessible to teachers, and sometimes teacher practice did not reflect best practice. Now this knowledge is used to design the analytics of online courses for students. For example if an online reading programme has analytics to notice whether students answer questions that require inferencing it can present learning activities to students who have been getting these questions wrong. Just like all effective learning activities, these online ones must be engaging and worthwhile, and the feedback needs to help learners know where they are at, where they are going and how they are going. These programmes are designed to be part of a rich learning experience for students (i.e. they do not replace the teacher). Lovett, Meyer and Thistle (2008) (PDF, 1.7 MB) found in their study that “OLI-Statistics students [blended learning] learned a full semester’s worth of material in half as much time and performed as well or better than students learning from traditional instruction over a full semester.”
2. Educators and researchers can develop a shared knowledge base when analytics are developed in collaborative discussions about the intent of actions, who determines what the outcomes are, and who receives what feedback can support. For example Tony Bryk’s Carnegie Foundation team of researchers help teachers discuss the assumptions associated with the question Can we tell from your digital profile whether you are at risk of learning and achieving? They reduced complex literature into driver maps, developed related analytics and tested personal theories. One collaboration found that the most powerful predictor of student success in College remedial mathematics courses is students’ sense of belonging to their mathematics class. The teachers have now been trialling different ways to improve students’ sense of belonging.
3. Analytics can be used to shift from the ‘easier’ to measure to the important. As educators we know the importance of wellbeing, capabilities, competencies and mindsets; the characteristics of 21C learners and leaders, but we have not had ways of knowing these things. Now the technology will allow us to focus on what is important. Learning organisations will be able to provide parents and whānau information about their young person that is important to them. Literacy and mathematical achievement can be one of the indicators of learning and achievement rather than the only indicators. For example, since 2009 Danish exams have analytics to assess students ability to explore and sift information for sense making therefore students need to use the internet to undertake the exams. New Zealand’s qualification authority has a programme to shift from internal and external assessment to one that emphasises assessment for learning.