Case Studies Evaluation of School 21’s Oracy Improvement Pilot for the Education Endowment Fund (EEF)

Schools UK Evaluation and Research

Challenge

AlphaPlus were commissioned by the Education Endowment Fund (EEF) to undertake an evaluation of School 21’s oracy improvement programme pilot.

School 21, supported by EEF, is working with the University of Cambridge to help Year 7 pupils improve their oracy, by embedding it in every lesson, i.e. their use of speech to express their thoughts and communicate effectively. The aim of the programme was to improve student’s oracy skills with the expectation that this would improve wider academic outcomes. The pilot ran from September 2016 until Summer 2017.


Solution

The pilot and the evaluation aimed to establish evidence to support:

  • the theory of change
  • the feasibility
  • the readiness for trial of the School 21 oracy model and ‘package’ in a range of different schools.

These three aims were measured in a number of ways:

  • Oracy assessment pre- and post-intervention attainment outcomes
  • Interview data
  • Pilot school teacher perceptions of change in pupils’ overall attainment – progress in different subjects (interview data)
  • Pilot school oracy lead perception questionnaire
  • Demographic and attainment data
  • Oracy lead questionnaire
  • Analysis of oracy assessment ‘reliability’ and conditions required for widespread use
  • Analysis of ‘reliability’ of oracy assessment tool
  • Analysis of most appropriate area(s) of the curriculum to detect a gain in attainment

The findings were presented in the final report on the pilot and were used to help to inform EEF’s decision about whether or not to go to trial, i.e. to undertake a full impact study.

The Project final report is available here.

Our approach to evaluation is to work initially with the School 21 oracy programme team to explicate the theory of change underpinning the intervention and articulate it through a logic model.


Our role

  • Mixed methods evaluation
  • Interviews with school leaders, oracy leads and teachers
  • Questionnaires
  • Analysis of work schemes
  • Analysis of performance data