KS5 England - Which benchmark should I use? August 2023

KS5 England - Which benchmark should I use? August 2023

In this article we outline the composition of the two benchmark sets available in Connect and Summit for Key Stage 5 England from 28 August 2023 onwards. 
It is designed to be read by Trust Leaders, Senior Leaders in schools and colleges and Leaders in Local Authorities. 

The situation we find ourselves in post-pandemic is an incredibly complex one as you will already have experienced. The article will take you through the value-added narrative which has resulted from the CAGs and TAGs years, and present you with options as to how you analyse your post-pandemic progress.

We urge you to read the whole article, as there are important implications for the use of the benchmark tools in the Connect and Summit platforms. It is not a simple solution, but we fee; it important to give you flexibility in making decisions over how you choose to look at your data. 

Headline summary:

  1. The value-added context for your Year 13 this summer is unique. We recommend using the client benchmarks in Connect and Summit when making judgements on strategic priorities from the 2023 results. 
  2. The context for monitoring, and for target setting, for incoming Year 13 students is complex. Analysis is likely to fall somewhere between the 2019 national and the 2023 Alps client benchmark. We recommend using the 2019 national benchmark with this cohort as it is likely to err on the side of caution. More below.
  3. The incoming Year 12 cohort are likely to see the value-added context return to one similar to the 2019 national dataset. We suggest that the 2019 benchmark is the most relevant to this cohort. 
  4. The way in which our benchmark tools have to operate leaves you with some decisions as to whether you switch your benchmark, and when you switch. 

National context 2019 to 2025

2019 - Pre- pandemic

The value-added analysis in Connect and Summit for KS5 is currently based on the national dataset released by the DfE in 2019. Since the pandemic, we have continued to use this 2019 benchmark to provide a consistent baseline to support you in making judgements on your school/college improvement priorities.


2022 - the CAGs cohort.

The value-added picture changed. We knew that:

  1. Students in that Year 13 cohort, the 2021.22 examination cohort, had elevated CAGs as their prior attainment 
  2. Examination outcomes for 2022 were elevated above the 2019 standards
This made us consider the accuracy of any improvement priorities determined using the 2019 benchmarks. We questioned whether the upshift in prior attainment and outcomes might actually result in a value-added pattern similar to 2019. We used the data submitted to Alps for the 2022 outcomes to generate an Alps client benchmark. This benchmark was available to Connect users to allow for comparisons between value-added outcomes for the 2019 national dataset against this Alps client subset for 2022. 

Comparison of the benchmarks revealed that the Alps client 2022 value-added thermometer for the strategic Quality Indicator was in line with the 2019 national. Although there was variance in subject thermometers, there was a general fit between the two benchmarks.  

The 2023 context - the TAGs cohort

The outcomes from the 2023 examination series present another unique set of challenges for colleagues:

  1. The examination standards effectively returned to pre-pandemic levels. This brings them back into line with 2019 outcomes
  2. There was uplift in the prior attainment for the 2023 KS5 cohort based on the awarded TAGs in 2021.

This presents a very different set of questions to those posed after the 2022 results. In 2023, there is no doubt that the value-added picture is not in line with that of 2019. Therefore a note of caution: your KS5 (A level and BTEC) value-added progress is likely to be less 'red' when viewing results against the 2019 dataset.

We have released the A level client benchmark into Connect and Summit and we recommend that you use the switch benchmark tools to agree your strategic priorities based on value-added gradings from the 2023 results. 
The BTEC 2016 client benchmark will be available from Monday 11 September 2023.

      Allowing staff to view analysis against the client benchmark:
      Should you decide that you would like to use the Alps client benchmark for the analysis of 2023 outcomes, then there are a few options to consider: 
  1. You can switch your benchmark for all users across your Connect or Summit account. This means that the Alps client benchmark could become your default for a period of time. Admin users in your Connect account have the rights to choose the benchmark under the Account Benchmark Selector page in Analysis Settings.
  2. You can give access to selected users only - this is the Session Benchmark toggle and is also available in Analysis Settings. Admin users will have to give view rights to selected staff before they can see this tool. It is a more flexible selector tool and changes the benchmark for the duration of one session at a time. 
  3. More details can be found here:  https://support.alps.education/portal/en/kb/articles/using-the-2023-client-benchmarks-in-connect-june-2023
  4. The Alps client benchmark is a subset of the national dataset. It will have gaps as there will be subject where the generation of thermometers is not possible due to small entry numbers, eg some MFL subjects. We have made the decision to allow you to switch benchmarks for school improvement purposes and not for accountability reasons. You should therefore use this new benchmark with some caution and in the spirit of school improvement for which it is intended. 

2024 context - the GCSE+ cohort

Your incoming Year 13 students will also have a unique value-added profile: 
  1. Students in Year 13 for 2023.24 sat GCSE examinations in 2022. Outcomes were elevated, therefore their prior attainment is likely to be elevated.  
  2. Examination outcomes for this cohort will return to 2019 standards. 
There are some crucial decisions to make over the most suitable benchmark from which to monitor this cohort of students. In the bullet points below, we summarise the facts:

  1. The value-added context for this cohort is likely to sit somewhere between the 2019 national progress benchmark and the 2023 Alps client benchmark. 
  2. Either benchmark might be argued to be the most relevant to this cohort of students in terms of MEG target setting and subsequent monitoring. 
  3. Using the 2019 national benchmark: 
    1. Students in Year 13 may be in higher prior attainment bands than they might have been in 2019, and therefore Minimum Expected Points and MEGs may be a grade to a split grade above where you might expect, depending on Alps band. 
    2. Monitoring against the 2019 benchmark is likely to result in your tracking being slightly more 'blue' than it may end up when compared to Alps client benchmarks which will be generated from 2024 results.
    3. You will have been monitoring these students against the 2019 national MEGs since September 2022.
  4. Using the 2023 Alps client benchmark: 
    1. Students may have slightly lower Minimum Expected Points and MEGs than they might have had, due to the fact that the 2022 GCSE outcomes were not as elevated as the TAGs, which is the prior attainment baseline for the 2023 client benchmark.
    2.  Monitoring against the 2023 client benchmark may result in your tracking being slightly on the 'red' or more optimistic side. We might argue that this is less helpful when it comes to targeted support and intervention, and in reporting to Governors and Trustees. 
    3. There are missing subject benchmarks in the client dataset due to smaller numbers. 
    4. If you switch to the Alps client MEGs, you might argue that you are reducing aspiration and sending negative messages to students. 
Taking all factors into account, we advise that you seriously consider using the 2019 national benchmarks to track Year 13 across 2023.24. We will generate a new client benchmark in the days following the 2024 examination results, and we would expect that you may see your overall grades improve slightly. We would consider that this may be a more palatable situation than using the 2023 client benchmark and having Alps grades drop if analysed against the 2024 client benchmark. 

      Target setting and MEG comparison 
  1. You will be able to generate MEGs from both the national and Alps client benchmark - simply switch to the appropriate benchmark and export your MEGs.
  2. However should you wish to compare student MEGs for Year 13, you can download and use the spreadsheet attached to this article - scroll to the bottom. 

      Using the benchmark tools for monitoring - points to consider:
  1. The 2023 client benchmark will be applied to your 2023 gradepoints across both Year 12 and 13 for monitoring purposes. It will also be applied to KS4 cohorts where you have a subscription.
  2. This means that if you use the Account Benchmark Selector to permanently switch your account, you would be switching benchmarks for every cohort, Year 13 - 10.
  3. You are likely to want to monitor the Year 12 and KS4 cohorts against the 2019 national benchmark and not the client benchmark.
  4. This will mean you will have to take care in when you use the Account Benchmark Selector and what this might be set to at any given point in your Assessment cycle. We recommend that you may only want to use this when asking colleagues to analyse their 2023 outcomes. 
  5. Remember that there is also the option of the Session Benchmark toggle. You would have to ensure that permissions were given to colleagues you wish to be able to use the toggle. Note that this individual user toggle will only apply for the duration of the session, and a users would have to re-toggle on logging back into Connect or Summit. 

The 2025 and beyond context

Your incoming Year 12 students are set to return to a profile similar to 2019: 
  1. Students in Year 12 for 2024.25 sat GCSE examinations in 2023 where standard returned to 2019 levels.  
  2. Examination outcomes for this cohort will also return to 2019 standards.

Therefore, the 2019 national dataset would be the most likely choice when monitoring this cohort across their KS5 courses. 
As discussed in points 3 and 4 above, this leaves you with some decisions to make about the benchmark you choose across your account, and when you choose to switch benchmarks, if at all. 


Summary of value-added contexts





 
The last piece of the puzzle - composition of each benchmark set by cohort. 
The tables below show which datasets are applied to which benchmark for which Academic Year. 
 
  1. When using the national 2019 benchmark: This has been applied across all gradepoints, including monitoring and examinations from 2018/19 outcomes onwards. 
  2. When using the Alps client benchmark:
    1. All gradepoints up to and including examinations 2020/21 are analysed against the national 2019 datatset.  
    2. All gradepoints across 2021/22 will be analysed against the 2022 Alps client benchmark. 
    3. All gradepoints across 2022/23 and monitoring for 2023/24 will use the 2023 Alps client benchmark. 

This is complex, but the grid is necessary to support you in understanding the context in which you are viewing your analysis. The highlighted cells indicate our recommended benchmark for each cohort. 



    • Related Articles

    • Which benchmark should I use? KS5 ENGLAND - AUGUST 2024

      In this article we outline the composition of the two benchmark sets available in Connect and Summit for Key Stage 5 England from 22 August 2024 onwards. It is designed to be read by Trust Leaders, Senior Leaders in schools and colleges and Leaders ...

    • KS4 England - Which benchmark should I use? August 2023

      Your value-added analysis in Connect is currently based on the national dataset released by the DfE in 2019. Since the pandemic, we have continued to use this 2019 benchmark to provide a consistent baseline to support you in making judgements on your ...

    • Which benchmark should I use? A level WALES - AUGUST 2024

      In this article we outline the composition of the two benchmark sets available in Connect and Summit for Key Stage 5 Wales from 12 September 2024 onwards. It is designed to be read by Senior Leaders in schools and colleges and Leaders in Local ...

    • Using the switch benchmark tools in CONNECT and SUMMIT - AUGUST 2024

      Selecting and switching benchmarks in Connect and Summit This article will describe the ways in which you can select benchmarks on the platforms and view your analysis against either the national benchmark or the client benchmark. The benchmark ...

    • Alps Analysis of A level ENGLAND - AUGUST 2024

      2024 A Level Results in England This article presents an overview of the findings from the A level Alps provider benchmarks for England. It provides an overview of the national attainment picture before presenting a summary of the Alps provider ...