News & Reports

Gemma O’Brien, AlphaPlus’ National Programme Lead for Personalised Assessments in Wales,  introduces sample-based assessments, how they work and why they are a useful tool for policy makers looking for evidence to inform strategy and intervention.

What are sample-based assessments?

Sample-based approaches to educational monitoring are assessments designed to measure the educational achievements of a representative sample of students across a country or jurisdiction. Unlike large-scale standardised tests that assess every student, these assessments aim to provide data on overall educational quality and trends at the national level without testing every individual. This approach is cost-effective, manageable, and allows for in-depth data collection while minimising testing burdens.

What do these assessments typically measure?

Sample-based national monitoring assessments typically measure a range of skills and knowledge areas essential to understanding student achievement, educational quality, and curriculum effectiveness. While the specific focus varies by jurisdiction, these assessments generally target core academic subjects, higher-order skills, and competencies relevant to students’ long-term success.

Sample-based assessments are widely used around the world to monitor national educational progress. These assessments help countries track trends in student achievement, inform educational policies, and provide insights into curriculum effectiveness. Notably, such assessments are used in jurisdictions such as the United States, Australia and New Zealand, although with varying approaches:

JurisdictionAssessmentFeatures
United StatesNational Assessment of Educational Progress (NAEP)The NAEP is the largest ongoing sample-based assessment in the United States, administered by the National Center for Education Statistics (NCES). It measures student performance in subjects like reading, mathematics, science, and writing for students in grades 4, 8, and 12.NAEP results are used to track academic progress over time, make comparisons among states, and inform educational policy at the national and state levels.
AustraliaNational Assessment Program: Literacy and Numeracy (NAPLAN) Sample AssessmentsAustralia conducts sample-based assessments in specific curriculum areas such as science, information and communication technology (ICT), and civics and citizenship.These assessments provide insights into students’ proficiency in these domains, with results used to monitor trends and inform educational policy and curriculum design.
New ZealandNational Monitoring Study of Student Achievement (NMSSA)Australia conducts sample-based assessments in specific curriculum areas such as science, information and communication technology (ICT), and civics and citizenship. These assessments provide insights into students’ proficiency in these domains, with results used to monitor trends and inform educational policy and curriculum design.

What are the key benefits of sample-based assessment approaches?

These assessments use a statistically representative sample of students to estimate national performance levels, often based on demographics such as age, grade level, location, and socioeconomic factors.

For example, in a national assessment of 4th-grade mathematics, a sample of schools and classrooms from urban, suburban, and rural areas would be selected to mirror the national student population.

Sample-based assessments are typically conducted periodically (e.g., every few years), allowing for trend analysis. By administering similar or comparable tests over time, policymakers and educators can observe changes in educational outcomes, track the effectiveness of reforms, and identify areas needing improvement.

Sample-based assessments often align with national or international educational standards, helping to determine if students are meeting expected competencies. Results can sometimes be benchmarked against international comparative assessments like PISA or TIMSS to see how national performance compares globally. These assessments also employ a sample-based approach.

The results of a national sample-based monitoring programme can inform policy decisions and curriculum adjustments by identifying strengths and weaknesses in the educational system. Results help in allocating resources more effectively, guiding professional development, and shaping educational initiatives.

Since only a sample of students participate, the testing burden on schools, teachers, and students is significantly reduced. This approach can be especially valuable in large countries or in systems with limited testing resources.

Assessment design considerations

Purpose

As with any assessment, designing an effective and valid assessment is strongly linked to the clarity of the purpose of the assessment. At an early stage, it should be established whether the programme aims to measure national trends, evaluate the impact of curriculum changes, identify disparities or provide insights into specific skill areas. A further important consideration is the intended use of the assessment outcomes and data – will results be used to inform policy, guide curriculum adjustments, improve teaching and learning practices or address equity issues? All decisions about assessment design must be driven by the assessment purpose to ensure validity.

Sampling strategy

At the outset, it will be necessary to determine an appropriate sample size that balances cost-effectiveness and manageability considerations with the need for robust and reliable data. A sufficiently large and diverse sample allows for robust statistical analysis while ensuring that key groups (e.g. particular demographic groups) are represented and that it is possible to stratify by characteristics such as school type, region or socioeconomic status if the assessment programme is intended to measure such aspects.

A related consideration is to consider potential sources of bias in the sampling or test design. An important aspect of statistical analysis will be to ensure that items are fair to all test-takers.

Other considerations

In addition to academic skills, some sample-based assessments incorporate surveys or questionnaires that assess students’ attitudes toward learning, motivation, self-efficacy, and social-emotional skills such as perseverance, teamwork, and empathy.

These measures are typically collected through student self-reports or teacher surveys, and they provide valuable insights into non-cognitive factors that impact learning and academic performance.

AlphaPlus and sample-based assessment

At AlphaPlus, we have extensive experience designing and delivering approaches for sample-based assessments, both in the UK and internationally. In addition, we also have the expertise to digitise these assessments, and analyse and report assessment data so that it is accessible and useful to policymakers, and where appropriate teachers and learners.  

AlphaPlus is part of AQA Global Assessment Services. AQA are the UK’s leading examination board, with over one million learners completing AQA qualifications annually. This combined experience makes us experts in the practical application of sample-based assessments.

If you would like support constructing a sample-based assessment approach or you have any questions about this article or how else we can support you, please get in touch with us here.

Gemma O’Brien – National Programme Lead for Personalised Assessments in Wales