Towards Analytics for Wholistic School Improvement: Hierarchical Process Modelling and Evidence Visualization

Ruth Elizabeth Deakin Crick
Simon Knight
Steven Barr


Central to the mission of most educational institutions is the task of preparing the next generation of citizens to contribute to society. Schools, colleges, and universities value a range of outcomes — e.g., problem solving, creativity, collaboration, citizenship, service to community — as well as academic outcomes in traditional subjects. Often referred to as “wider outcomes,” these are hard to quantify. While new kinds of monitoring technologies and public datasets expand the possibilities for quantifying these indices, we need ways to bring that data together to support sense-making and decision-making. Taking a systems perspective, the hierarchical process modelling (HPM) approach and the “Perimeta” visual analytic provides a dashboard that informs leadership decision-making with heterogeneous, often incomplete evidence. We report a prototype of Perimeta modelling from education, aggregating wider outcomes data across a network of schools, and calculating their cumulative contribution to key performance indicators, using the visual analytic of the Italian flag to make explicit not only the supporting evidence, but also the challenging evidence, as well as areas of uncertainty. We discuss the nature of the modelling decisions and implicit values involved in quantifying these kinds of educational outcomes.


Learning analytics; academic analytics; leadership; decision support; complex systems; educational values; visualization; dashboard; uncertainty; risk; surveys

Full Text:



Assessment Reform Group. (1999). Assessment for learning: Beyond the black box. Cambridge, UK: University of Cambridge School of Education.

Assessment Reform Group. (2002). Testing, motivation and learning. Cambridge, UK: Assessment Reform Group.

Beer, S. (1984). The viable system model: Its provenance, development, methodology and pathology. Journal of the Operational Research Society, 35(1), 7–25.

Beer, S. (1985). Diagnosing the system for organisations. Hoboken, NJ: John Wiley.

Behrens, J., & DiCerbo, K. (2014). Harnessing the currents of the digital ocean. In J. Larussen & B. White (Eds.), Learning analytics: From research to practice. New York: Springer.

Blackwell, L. S., Trzesniewski, K. H., & Dweck, C. S. (2007). Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child Development, 78(1), 246–263.

Blockley, D. (2010). The importance of being process. Civil Engineering and Environmental Systems, 27(3), 189–199.

Blockley, D., & Godfrey, P. (2000). Doing it differently: Systems for rethinking construction. London: Telford.

Bottery, M. (2004). The challenges of educational leadership. London: Paul Chapman.

Bower, D. F. (2004). Leadership and the self-organizing school. Paper presented at the Complexity Science and Educational Research Conference, Kingston, Ontario, Canada.

Bryk, A., & Gomez, L. (2008). Ruminations on Reinventing an R&D Capacity for Educational Improvement. Paper presented at the The Supply Side of School Reform and the Future of Educational Entrepreneurship, Stanford, CA.

Bryk, A. S., Sebring, P. B., Allensworth, E., Luppescu, S., & Easton, J. Q. (2009). Organizing schools for improvement: Lessons from Chicago: University of Chicago Press.

Bryk, A., Gomez, L. M., & Grunow, A. (2011). Getting ideas into action: Building networked improvement communities in education. In M. T. Hallinan (Ed.), Frontiers in Sociology of Education (pp. 127–162). Netherlands: Springer.

Bryk, A., Gomez, L., Grunow, A., & LeMahieu, P. (2015). Learning to improve: How America’s schools can get better at getting better. Cambridge, MA: Harvard Educational Press.

Bryk, A., Sebring, P., Allensworth, E., Luppescu, S., & Easton, J. (2010). Organizing schools for improvement: Lessons from Chicago. Chicago, IL: University of Chicago Press.

Buckingham Shum, S., & Deakin Crick, R. (2012). Learning dispositions and transferable competencies: Pedagogy, modelling and learning analytics. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge (LAK ʼ12), 29 April–2 May 2012, Vancouver, BC, Canada (pp. 92–101). New York: ACM.

Buckingham Shum, S., & Deakin Crick, R. (2016). Learning analytics for 21st century competencies. Journal of Learning Analytics and Knowledge, 3(2).

Campbell, J., DeBlois, P., & Oblinger, D. (2007). Academic analytics: A new tool for a new era. Educause Review, 42(4), 40–57.

Checkland, P. (1999). Systems thinking systems practice. Chichester, UK: John Wiley.

Checkland, P., & Scholes, J. (1999). Soft systems methodology in action. Chichester, UK: John Wiley.

Claxton, G. (2008). Expanding the capacity to learn: A new end for education? Opening Keynote Address, British Educational Research Association Annual Conference, 6 September 2006, Warwick University.

Cui, W., & Blockley, D. (1990). Interval probability theory for evidential support. International Journal of Intelligent Systems, 5(2), 183–192.

Daniel, B. (2015). Big Data and analytics in higher education: Opportunities and challenges. British Journal of Educational Technology, 46(5), 904–920.

Darling-Hammond, L., & Bransford, J. (2005). Preparing teachers for a changing world: What teachers should learn and be able to do. San Francisco, CA: Jossey-Bass.

Davis, J., MacDonald, A., & White, L. (2010). Problem-structuring methods and project management: an example of stakeholder involvement using Hierarchical Process Modelling methodology. Journal of the Operational Research Society,, 61, 893-904.

Davis, B., & Sumara, D. (2006). Complexity and education inquiries into learning, teaching and research. London: Routledge

Davis, J., & Fletcher, P. (2000). Managing assets under uncertainty. Society of Petroleum Engineers Asia Pacific Conference on Integrated Modelling for Asset Management (APCIMAM 2000), 25-26 April 2000, Yokohama, Japan.

Deakin Crick, R. (2017). Learning analytics: Layers, loops and processes in a virtual learning infrastructure. In G. Siemens & C. Lang (Eds.), Handbook of learning analytics & educational data mining. Society of Learning Analytics Research (SoLAR).

Deakin Crick, R., Barr, S., Green, H., & Pedder, D. (2015). Evaluating the wider outcomes of schools: Complex systems modelling. Educational Management Administration and Leadership, 45(4).

Deakin Crick, R., Huang, S., Ahmed Shafi, A., & Goldspink, C. (2015). Developing resilient agency in learning: The internal structure of learning power. British Journal of Educational Studies, 63(2), 121–160.

Dempster, A. (1969). Elements of continuous multivariate analysis. Reading, MA: Addison-Wesley.

Delors, J. (2000). Learning: The treasure within.

Dweck, C. S. (2000). Self-theories: Their role in motivation, personality, and development. New York: Psychology Press.

European Commission. (2007). Key competences for lifelong learning: European reference framework. Luxembourg: Publications Office of the European Union.

Ferguson, R. (2012). Learning analytics: Drivers, developments and challenges. International Journal of Technology Enhanced Learning, 4(5/6), 304–317.

Ferguson, R., Clow, D., Macfadyen, L., Essa, A., Dawson, S., & Alexander, S. (2014). Setting learning analytics in context: Overcoming the barriers to large-scale adoption. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 251–253). New York: ACM.

Goldspink, C. (2015). Communities making a difference national partnerships SA teaching for effective learning pedagogy research project 2010–2013. Adelaide: South Australia Department for Education and Child Development.

Goldspink, C., & Foster, M. (2013). A conceptual model and set of instruments for measuring student engagement in learning. Cambridge Journal of Education, 43(3), 291–311.

Gunter, H. (2001). Leaders and leadership in education. London: Paul Chapman.

Hall, J., Le Masurier, J., Baker-Langman, E., Davis, J., & Taylor, C. (2004). Decision-support methodology for performance-based asset management. Journal of Civil Engineering and Environmental Systems, 21, 51–75.

Hall, J., Blockley, D., & Davis, J. (1998). Uncertain inference using interval probability theory. International Journal of Approximate Reasoning, 19, 247–264.

Harlen, W., & Deakin Crick, R. (2003). Testing and motivation for learning. Assessment in Education, 10(2), 169–207.

James, M., & Gipps, C. (1998). Broadening the basis of assessment to prevent the narrowing of learning. The Curriculum Journal, 9(3), 285–297.

James, M., McCormick, R., Black, P., Carmichael, P., Drummond, M., Fox, A., Wiliam, D. (2007). Improving learning how to learn: Classrooms, schools and networks. London: Routledge.

Joldersma, C., & Deakin Crick, R. (2010). Citizenship, discourse ethics and an emancipatory model of lifelong learning. London: Routledge.

Jackson, M. C. (2000). Systems approaches to management. London: Kluwer Academic.

Knight, S., Buckingham Shum, S., & Littleton, K. (2014). Epistemology, assessment, pedagogy: Where learning meets analytics in the middle space. Journal of Learning Analytics, 1(2), 23–47.

Norris, D., Baer, L., & Offerman, M. (2009). A national agenda for action analytics, White Paper. National Symposium on Action Analytics, 21–23 September 2009, St. Paul, Minnesota, USA.

MacBeath, J., & Cheng, Y. (2008). Leadership for learning: International perspectives. London: Routledge.

MacBeath, J., & McGlynn, A. (2002). Self-evaluation: What’s in it for schools? London: RoutledgeFalmer.

Mason, M. (Ed.). (2008). Complexity theory and the philosophy of education. Chichester, UK: Wiley-Blackwell.

McCombs, B., & Miller, L. (2008). A school leader’s guide to creating learner-centered education: From complexity to simplicity. New York: Corwin Press.

McCombs, B. L., Daniels, D. H., & Perry, K. E. (2008). Children’s and teachers’ perceptions of learner-centered practices, and student motivation: Implications for early schooling. The Elementary School Journal, 109(1), 16–35.

Macfadyen, L., Dawson, S., Abelardo, P., & Gašević, D. (2014). Embracing big data in complex educational systems: The learning analytics imperative and the policy challenge. Research & Practice in Assessment, 9, 17–28.

McKinsey Report. (2007). How the world’s best performing school systems come out on top.

OECD. (2001). Investing in competencies for all. Meeting of OECD Education Ministers. Retrieved from

Opfer, V., & Pedder, D. (2011). The lost promise of teacher professional development in England. European Journal of Teacher Education, 34(1), 3–24.

Pedder, D. (2006). Organizational conditions that foster successful classroom promotion of learning how to learn. Research Papers in Education, 21(2), 171–200.

Pedder, D. (2007). Profiling teachers’ professional learning practices and values: Differences between and within schools. The Curriculum Journal, 18(3), 231–252.

Pedder, D., James, M., & MacBeath, J. (2005). How teachers value and practise professional learning. Research Papers in Education, 20(3), 209–243.

Piety, P. J., Hickey, D. T., & Bishop, M. J. (2014). Educational data sciences: Framing emergent practices for analytics of learning, organizations, and systems. Proceedings of the 4th International Conference on Learning Analytics and Knowledge (LAK ʼ14), 24–28 March 2014, Indianapolis, IN, USA (pp. 193–202). New York: ACM.

Reay, D. W. D. (1999). ‘I'll be a nothing’: Structure, agency and the construction of identity through assessment. British Educational Research Journal, 25(3), 343–354.

Ritchie, R., & Deakin Crick, R. (2007). Distributing leadership for personalising learning. London: Continuum.

Robinson, V., Lloyd, C., & Rowe, K. (2009). The impact of leadership on student outcomes: An analysis of the differential effects of leadership types. Educational Administration Quarterly, 45(3), 515–520.

Rychen, D., & Salagnik, L. (2001). Definition and selection of key competencies. Theoretical and Conceptual Foundations (DeSeCo) Background paper, Swiss Federal Statistical Office, OECD, ESSI, Neuchatel.

Selvin, A., & Buckingham Shum, S. (2014). Constructing knowledge art: An experiential perspective on crafting participatory representations. Williston, VT: Morgan & Claypool.

Shafer, G. (1976). A mathematical theory of evidence. Princeton, NJ: Princeton University Press.

Silins, H., & Mulford, B. (2004). Schools as learning organisations: Effects on teacher leadership and student outcomes. School Effectiveness and School Improvement, 15, 443–466.

Sillitto, H. (2015). Architecting systems: Concepts, principles and practice. London: College Publications Systems Series.

Snowden, D., & Boone, M. E. (2007, November). A leader’s framework for decision making. Harvard Business Review.

Thomas, D., & Seely Brown, J. (2009). Learning for a world of constant change: Homo sapiens, Homo faber & Homo ludens revisited. Paper presented at the 7th Glion Colloquium, June 2009, University of Southern California.

Thomas, D., & Seely Brown, J. (2011). A new culture of learning cultivating the imagination for a world of constant change. CreateSpace Independent Publishing Platform.

West-Burnham, J. (2005). Leadership for personalizing learning. In S. De Frietas & C. Yapp (Eds.), Personalizing learning in the 21st century. Stafford, UK: Network Educational Press.

White, L. 2006. Evaluating problem-structuring methods: Developing an approach to show the value and effectiveness of PSMs. Journal of the Operational Research Society, 57, 842–845.

Yeager, D., Bryk, A., Muhich, J., Hausman, H., & Morales, L. (2015). Practical measurement. San Francisco, CA: Carnegie Foundation for the Advancement of Teaching.

Zohar, D. (1997). Rewiring the corporate brain. San Francisco, CA: Berrett-Koehler Publishers.



  • There are currently no refbacks.