Which benchmark should I use? A level WALES - AUGUST 2024

Which benchmark should I use? A level WALES - AUGUST 2024

In this article we outline the composition of the two benchmark sets available in Connect and Summit for Key Stage 5 Wales from 12 September 2024 onwards. 
It is designed to be read by Senior Leaders in schools and colleges and Leaders in Local Authorities. 

The article will take you through the value-added narrative which has resulted from the pandemic years, and present you with options as to how you analyse your post-pandemic progress.

We urge you to read the whole article, as there are important implications for the use of the benchmark tools in the Connect and Summit platforms. It is not a simple solution, but we feel it important to give you flexibility in making decisions over how you choose to look at your data. 

      Headline summary:

  1. Examination analysis: The value-added context for your Year 13 this summer is unique. We recommend using the provider benchmarks in Connect and Summit when making judgements on strategic priorities from the 2024 results. 
  2. Target setting and monitoring in September: incoming Year 13 students are likely to see the value-added context return to one similar to the 2019 national dataset. We suggest that the 2019 benchmark is the most relevant to this cohort. 
  3. The way in which our benchmark tools have to operate leaves you with some decisions as to whether you switch your benchmark, and when you switch. 

      National context 2019 to 2025

2019 - Pre- pandemic

The default value-added analysis in Connect and Summit for KS5 is currently based on the national dataset from 2019. Since the pandemic, we have continued to use this 2019 benchmark to provide a consistent baseline to support you in making judgements on your school/college improvement priorities.

      2024 examination context - the GCSE+ cohort

Your outgoing Year 13 students have a unique value-added profile: 
  1. Students in Year 13 for 2023.24 sat GCSE examinations in 2022. Outcomes were elevated, therefore their prior attainment is likely to be elevated.  
  2. Examination outcomes for this cohort have returned towards 2019 standards, although still slightly elevated. 
As their situation is unique, we recommend that the 2024 Alps provider benchmark will represent the fairest comparative benchmark on which to base your analysis of results, and on which to base your priorities going forward. Using the 2019 default benchmark for this cohort is likely to result in your analysis being slightly more 'blue' than it may end up when compared to Alps provider benchmarks which will be generated from 2024 results.

The Alps provider benchmark will be available in Connect and Summit on 12th September. A reminder that this benchmark will be comprised of Alps Providers in Wales only. 

      2025 cohort - incoming Year 13

This cohort also have a prior attainment that is elevated compared to 2019 levels. Their GCSEs were in 2023 where GCSEs results were at the midpoint between 2022 and 2019.  However their GCSE results were not as elevated as this summers Year 13 cohort. 
There are some crucial decisions to make over the most suitable benchmark from which to monitor this cohort of students. In the bullet points below, we summarise the facts:

  1. The value-added context for this cohort is likely to sit somewhere between the 2019 national progress benchmark and the 2024 Alps provider benchmark. 
  2. Either benchmark might be argued to be the most relevant to this cohort of students in terms of MEG target setting and subsequent monitoring. 
  3. Using the 2019 national benchmark: 
    1. Students in incoming Year 13 may be in higher prior attainment bands than they might have been in 2019, and therefore Minimum Expected Points and MEGs may be a grade to a split grade above where you might expect, depending on Alps band. 
    2. Monitoring against the 2019 benchmark is likely to result in your tracking being slightly more 'blue' than it may end up when compared to Alps provider benchmarks which will be generated from 2025 results.
    3. You will have been monitoring these students against the 2019 national MEGs since September 2023
  4. Using the 2024 Alps provider benchmark: 
    2. Students may have slightly lower Minimum Expected Points and MEGs than they might have had, due to the fact that the 2023 GCSE outcomes were not as elevated as those in 2022, which is the prior attainment baseline for the 2024 provider benchmark.
    3.  Monitoring against the 2024 provider benchmark may result in your tracking being slightly on the 'red' or more optimistic side. We might argue that this is less helpful when it comes to targeted support and intervention, and in reporting to Governors. 
    4. There are missing subject benchmarks in the client dataset due to smaller numbers. 
    5. If you switch to the Alps provider MEGs, you might argue that you are reducing aspiration and sending negative messages to students. 
Taking all factors into account, we advise that you seriously consider using the 2019 national benchmarks to track Year 13 across 2024.25. We will generate a new provider benchmark in the days following the 2025 examination results, and we would expect that you may see your overall grades improve slightly. We would consider that this may be a more palatable situation than using the 2024 provider benchmark and having Alps grades drop if analysed against the 2025 provider benchmark. 

Target setting and MEG comparison

You will be able to generate MEGs from both the national and Alps provider benchmark - simply switch to the appropriate benchmark and export your MEGs.

 Using the benchmark tools for monitoring - points to consider:

  1. The 2024 provider benchmark will be applied to your 2024/25 gradepoints across both Year 12 and 13 for monitoring purposes. It will also be applied to KS4 cohorts where you have a subscription.
  2. This means that if you use the Account Benchmark Selector to permanently switch your account, you would be switching benchmarks for every cohort, Year 13 - 10.
  3. You are likely to want to monitor the Year 12 and KS4 cohorts against the 2019 national benchmark and not the provider benchmark.
  4. This will mean you will have to take care in when you use the Account Benchmark Selector and what this might be set to at any given point in your Assessment cycle. We recommend that you may only want to use this when asking colleagues to analyse their 2024 outcomes. 
  5. Remember that there is also the option of the Session Benchmark toggle. You would have to ensure that permissions were given to colleagues you wish to be able to use the toggle. Note that this individual user toggle will only apply for the duration of the session, and a users would have to re-toggle on logging back into Connect or Summit. 

The 2026 and beyond context

Your incoming Year 12 students are set to return to a profile similar to 2019: 
  1. Students in Year 12 for 2024.25 sat GCSE examinations in 2024 where standards are set to return to 2019 levels.  
  2. Examination outcomes for this cohort will also return towards 2019 standards.
Therefore, the 2019 national dataset would be the most likely choice when monitoring this cohort across their KS5 courses. 
As discussed in points 3 and 4 above, this leaves you with some decisions to make about the benchmark you choose across your account, and when you choose to switch benchmarks, if at all. 

Summary of value-added contexts





 
The last piece of the puzzle - composition of each benchmark set by cohort. 
The tables below show which datasets are applied to which benchmark for which Academic Year. 
 
  1. When using the national 2019 benchmark: This has been applied across all gradepoints, including monitoring and examinations from 2018/19 outcomes onwards. 
  2. When using the Alps provider benchmark:
    1. All gradepoints up to and including examinations 2020/21 are analysed against the national 2019 datatset.  
    2. All gradepoints across 2021/22 will be analysed against the 2022 Alps provider benchmark. 
    3. All gradepoints across 2022/23 will be analysed against the 2023 Alps provider benchmark. 
    4. All gradepoints across 2023/2024 and beyond (monitoring for 2024/25 will use the 2024 Alps provider benchmark. 

This is complex, but the grid is necessary to support you in understanding the context in which you are viewing your analysis. The highlighted cells indicate our recommended benchmark for each cohort. 




    • Related Articles

    • Alps START – Start of Term Analysis, Results and Trends - WALES ONLY - June 2024

      Alps START – Start of Term Analysis, Results and Trends Get your data to us in Results weeks for benchmarks based on 2023 Alps client data In September 2022, we provided schools and colleges in England with A level analysis based on data submitted to ...

    • Which benchmark should I use? KS5 ENGLAND - AUGUST 2024

      In this article we outline the composition of the two benchmark sets available in Connect and Summit for Key Stage 5 England from 22 August 2024 onwards. It is designed to be read by Trust Leaders, Senior Leaders in schools and colleges and Leaders ...

    • KS5 England - Which benchmark should I use? August 2023

      In this article we outline the composition of the two benchmark sets available in Connect and Summit for Key Stage 5 England from 28 August 2023 onwards. It is designed to be read by Trust Leaders, Senior Leaders in schools and colleges and Leaders ...

    • Using the switch benchmark tools in CONNECT and SUMMIT - AUGUST 2024

      Selecting and switching benchmarks in Connect and Summit This article will describe the ways in which you can select benchmarks on the platforms and view your analysis against either the national benchmark or the client benchmark. The benchmark ...

    • KS4 England - Which benchmark should I use? August 2023

      Your value-added analysis in Connect is currently based on the national dataset released by the DfE in 2019. Since the pandemic, we have continued to use this 2019 benchmark to provide a consistent baseline to support you in making judgements on your ...