EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Methodology
---
We compare sea ice concentrations from the [CMIP6](https://cds.climate.copernicus.eu/datasets/projections-cmip6?tab=overview) historical experiment with that obtained from [passive microwave satellite products from EUMETSAT OSI- SAF and ESA CCI](https://cds.climate.copernicus.eu/datasets/satellite-sea-ice-concentration?tab=overview).  Time series of the evaluation metrics Integrated Ice Edge Error (IIEE) ([Goessling et al, 2016](https://doi.org/10.1002/2015GL067232); [Henke et al, 2023)](https://doi.org/10.1080/15230430.2023.2271592), bias in extent and area, and the root mean square error (RMSE) in sea ice concentration are produced and plotted. All of these statistics are area-weighted averages. We consider all the CMIP6 models that output sea ice concentration in the historical experiment, but remove outliers by only plotting the interquartile limits (IQLs). Other authors have chosen different subsetting approaches - e.g. by scoring the models and only retaining the top-performing ones. We recommend users test as many models as possible, before deciding if a subset adequately represents the uncertainty for their application.

Decadal climatologies for both the models and observations are also produced and compared, for the months March, September and December.

The “Analysis and results” section is organised as follows:

**[](section-1)**

**[](section-2)**

**[](section-3)**

        **[](section-3.1)**

        Plot time series of the error metrics.

        **[](section-3.2)**

        This section has maps of climatologies and their biases for the [](section-3.2.1), [](section-3.2.2), [](section-3.2.3), [](section-3.2.4). We also plot maps for the [](section-3.2.5) since this was an interesting month for the climate projections.


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.2 Set parameters
---
- Set the time period to be analysed with `year_start` and `year_stop`.
- Set the concentration threshold `sic_threshold` for determining sea ice extent (we use 30% to be consistent with the ice edge product).
- `clim_months` is a list of the months to plot the climatologies for.
- `decades_historical` of starting years of decades to calculate climatologies for.
- `projections` is a dictionary of projections to plot the climatology maps in.
- `map_slices` is a dictionary of slices in the x and y directions (on observation grid) to zoom in on the climatology maps.
- Set the regions to be analysed with the list `regions`.
- Set the models to be evaluated with the list `models`.

Sea Ice Concentration Threshold
months to plot the climatologies for
starting years of decades to calculate climatologies for
projections to plot the climatology maps in
slices in x and y directions (on observation grid) to zoom in on the climatology maps
Select regions
Choose CMIP6 historical models


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.4 Functions to create the time series
---
These functions are all applied to a single CMIP6 model.
- `interpolate_to_satellite_grid` interpolates the model to the satellite grid.
- `get_monthly_interpolated_data` is applied to both the model and satellite data, to take the monthly mean (the satellite data is daily, but this also forces all the models to have the same time coordinate) and interpolate to the satellite grid (only for model data). It also calculates the RMS error (only for satellite data).
- `get_satellite_data` downloads the daily satellite data and applies `get_monthly_interpolated_data` to get the monthly mean and the RMS error in the data.
- `compare_model_vs_obs` compares the sea ice concentration from a single CMIP6 model with satellite estimates, calculating error metrics like biases in concentration and extent, RMSE, and IIEE.
- `compute_sea_ice_evaluation_diagnostics` downloads the satellite and model data and calls `compare_model_vs_obs` to get a time series of error metrics.

Remove nan columns
Get variables
Compute useful variables
Compute output


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.5 Post-processing and plotting of the time series
---
- `postprocess_dataset` is applied after loading from the cache and renames some variables and dimensions for easier use.
- `plot_timeseries` plots time series for each error metric (sea ice concentration bias and RMSE, sea ice extent bias and IIEE), and for each region (Arctic or Antarctic).

err_name = "RMS obs error"
get median and interquartile limits to show the spread of the models
plot the median
Plot spread and add observation errors
add obs errors to plots
add obs errors to plots
fix time labels for last variable


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.6 Functions to compute the climatologies
---
- `compute_monthly_climatology` transforms a dataset (which can contain model or observation data) by sorting according to month, taking the time average, and interpolating to the satellite grid (model data only).
- For a specified time interval, `get_monthly_climatology_eumetsat` downloads the satellite data and calculates the climatology for the selected region and months.
- For a specified time interval, `get_monthly_climatology_model` downloads and calculates the climatology for the selected region, months and model.
- `get_monthly_climatologies_cmip6` calls `get_monthly_climatology_model` once for each model and calculates the climatology of the ensemble mean.

some models produce extra variables so drop any that are not needed


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.7 Post-processing and plotting of climatologies
---
- `postprocess_climatology` is applied after loading a climatology from the cache. It makes the name and units of the sea ice concentration variable consistent between datasets.
- `make_sic_maps` plots the climatology of the CMIP6 ensemble mean next to the observed climatology for the three decades considered (1985-1994, 1995-2004, 2005-2014), which are shown in rows.
- `make_sic_bias_maps` plots the bias (CMIP6 ensemble mean minus the observations) for the same decades.
- `compare_sic_maps` is a wrapper for those functions.

rename month
rename SIC and convert to %

(section-2)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 2. Download and transform data
---
Download and transform the data and save it to disk. The second time `download_and_transform` is run it will save time by loading the transformed data from the disk.


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 2. Download and transform data > 2.1 Set some parameters for downloading
---
- `interpolation_kwargs` sets some interpolation options.
- `io_kwargs` sets some parameters to speed up the download process.
- `eval_kwargs` contains all the options that will be passed to `download.download_and_transform` when creating the time series of evaluation metrics.

Parameters to speed up IO


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 2. Download and transform data > 2.2 Download and transform CMIP6 data for time series
---
For each model and region, we download the data, transform it with `compute_sea_ice_evaluation_diagnostics`, save it to disk and post-process with `postprocess_dataset`. The time series are then combined into one dataset with extra dimensions `model` and `region`.

options to be passed to compute_sea_ice_evaluation_diagnostics


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 2. Download and transform data > 2.4 Download the mean climatology for the historical CMIP6 experiment
---
We use all models in the historical experiment to get the ensemble mean and get the climatology for one region and decade at a time to see the spatial distribution of the bias when compared to satellite estimates, and to see how the bias develops with time.

set some options for plotting the climatologies

(section-3)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 3. Results > 3.1 Time series of evaluation metrics
---
The evalution metrics that will be the most important here are the IIEE and the bias in extent, as the determination of the presence or absence of ice from passive microwave is more reliable than the actual value of the concentration derived from it. In particular, there is systematic underestimation of the concentration. This is especially true in summer due to meltponds for example, but it is also true in winter where the EUMETSAT OSI-SAF product usually estimates about 90% concentration in the Arctic pack (when it is very close to 100%, as can be seen from SAR images for example). The ESA CCI product which has a smaller footprint usually estimates a higher concentration than EUMETSAF OSI-SAF in the Arctic pack. Nevertheless it is still interesting to look at the area-weighted bias and RMSE in concentration itself - for example an underestimation of concentration would generally be undesirable for a model. The plots show the median and inter-quartile limits (IQL) to show the spread of our ensemble of CMIP6 models (all those that output sea ice concentration in the historical experiment).

The IIEE for the CMIP6 models is quite high, being about $2\times10^6$km$^2$ in the Arctic and twice this in the Antarctic. (Refer to the sea ice diagnostics notebook which shows the historical Arctic minimum extent is about $8\times10^6$km$^2$.) The bias in Arctic extent is not too high, although it is quite variable. For Antarctica however it is quite high, underestimating it by about $4\times10^6$km$^2$ by 2015 (growing from around $1\times10^6$km$^2$ in 1980).

The bias for the concentration itself is quite low for CMIP6, being well within the limits of the observation error. There is a clear tendency for underestimation by CMIP6 in Antarctica though, with the bias dropping linearly. The RMS error for CMIP6 is also very high, indicating a different spatial distribution to the satellite data, since the bias was not too high.

We will take a look at the spatial distribution more closely in the next section.

Finally we note that when the ESA-CCI and EUMETSAT OSI-SAF products overlap in time (after 2002), the metrics for CMIP6 are quite similar.

(section-3.2)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 3. Results > 3.2 Spatial distribution of errors
---
We compare 10-year-averaged maps from 1985-2015 which is roughly the overlap period between the historical experiment and the EUMETSAT OSI-SAF product. In the Arctic, the mean concentration is quite close to the mean from observations, but there is disagreement in the seas with seaonal ice - e.g. Greenland, Barents, Labrador, Bering and Okhotsk Seas and Hudson Bay, and (in the summer) off the Russian and Alaskan coasts. For the Antarctic there is systematic underestimation of concentration.

(section-3.2.1)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 3. Results > 3.2.1 Arctic minimum
---
Looking at the maps below for September, we can see there is a general underestimation in the pack ice and an overestimation in the MIZ and near the coasts. This pattern persists with time although the locations move as the Arctic ice cover shrinks with time. This may be a resolution effect, since roughly the same amount of ice (low bias in sea ice concentration) may just be being spread out over a larger area.

(section-3.2.2)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 3. Results > 3.2.2 Arctic maximum
---
Looking at the maps below for March, we can see the ice in the central Arctic is generally unbiased, but there are some areas where there is strong underestimation - the Bering Sea and the Sea of Okhotsk - and there is strong overestimation in the Greenland Sea and the North Atlantic Ocean near the entrance to Hudson Bay. Overall, the total ice area is roughly the same in CMIP6 and the satellite observations.

(section-3.2.3)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 3. Results > 3.2.3 Antarctic minimum
---
There is consistently too little ice on the Atlantic side of Antarctica (in the Weddell, Bellingshausen and Amundsen Seas). The amount of sea ice in those areas is also decreasing with time, while the concentration from satellite is staying relatively constant. On the Pacific and Indian ocean sides there is more of a dipole pattern (too little at the coast and too much away from it), probably due to the effect of low resolution in the CMIP6 models.

(section-3.2.4)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 3. Results > 3.2.4 Antarctic maximum
---
For the month of September there is in general too little ice everywhere, although away from the coast at longitude about 140$^\circ$W in the Atlantic Ocean, and in the Indian Ocean the underestimation is most pronounced. The underestimation on those areas is also increasing with time, while the concentration from satellite is staying relatively constant in this month as well.

(section-3.2.5)=


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > Analysis and results > 3. Results > 3.2.5 Arctic December (relevant to the Arctic shipping route assessment)
---
One of the other CMIP6 quality assessments looks at projections of accessibility of Arctic shipping routes, where sea ice in December is particularly interesting. Hence we also plot some maps for this month here. The CMIP6 ensemble mean is quite close to the satellite data in the Arctic Ocean, with the main feature of the bias maps being the underestimation in the Hudson Bay. Note that not all models that provide  sea ice concentration in the historical experiment also provide it in the different projection experiments, so that adds some additional uncertainty to these conclusions.


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > ℹ If you want to know more > Key resources
---
Introductory sea ice materials:
- [Role of sea ice in the climate](https://marine.copernicus.eu/explainers/why-ocean-important/sea-ice)
- [Sea ice as an indicator of climate change](https://climate.copernicus.eu/climate-indicators/sea-ice#)
- [Observing sea ice with satellites](https://www.metoffice.gov.uk/research/climate/cryosphere-oceans/sea-ice/measure)

Introductory CMIP6 materials:
- [A short introduction to CMIP and CMIP6](https://www.wcrp-climate.org/wgcm-cmip/cmip-video)
- [CMIP6: the next generation of climate models explained](https://www.carbonbrief.org/cmip6-the-next-generation-of-climate-models-explained/)

Code libraries used:
- [C3S EQC custom functions](https://github.com/bopen/c3s-eqc-automatic-quality-control/tree/main/c3s_eqc_automatic_quality_control) developed by [B-Open](https://www.bopen.eu/)


EQC Quality Assessment: "Historical accuracy of sea ice extent in the CMIP6 experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q02  |  Category: Climate_Projections
Section: Historical accuracy of sea ice extent in the CMIP6 experiments > ℹ If you want to know more > References
---
1. Davy, R., and S. Outten (2020). The Arctic Surface Climate in CMIP6: Status and Developments since CMIP5. J. Climate, 33, 8047–8068,  https://doi.org/10.1175/JCLI-D-19-0990.1

2. Frankignoul, C., L. Raillard, B. Ferster, and Y. Kwon (2024). Arctic September sea ice concentration biases in CMIP6 models and their relationships with other model variables. J. Climate,  https://doi.org/10.1175/JCLI-D-23-0452.1 , in press.

3. Goessling, H. F., S. Tietsche, J. J. Day, E. Hawkins, and T. Jung (2016). Predictability of the Arctic sea ice edge, Geophys. Res. Lett., 43, 1642–1650,  https://doi.org/10.1002/2015GL067232

4. Henke, M., F. Cassalho, T. Miesse, C. M. Ferreira, J. Zhang and T. M. Ravens (2023). Assessment of Arctic sea ice and surface climate conditions in nine CMIP6 climate models. Arctic, Antarctic, and Alpine Research, 55(1).  https://doi.org/10.1080/15230430.2023.2271592

5. Nie, Y., X. Lin,  Q. Yang, J. Liu, D. Chen and P. Uotila (2023). Differences between the CMIP5 and CMIP6 Antarctic sea ice concentration budgets. Geophysical Research Letters, 50, e2023GL105265.  https://doi.org/10.1029/2023GL105265

6. Li, S., Y. Zhang, C. Chen, Y. Zhang, D. Xu, and S. Hu (2023). Assessment of Antarctic Sea Ice Cover in CMIP6 Prediction with Comparison to AMSR2 during 2015–2021. Remote Sensing 15(8): 2048,  https://doi.org/10.3390/rs15082048

7. Roach, L. A., J. Dörr, C. R. Holmes, F. Massonnet, E. W. Blockley, D. Notz, T. Rackow, M. N. Raphael, S. P. O'Farrell, D. A. Bailey, and C. M. Bitz (2020). Antarctic sea ice area in CMIP6. Geophysical Research Letters, 47, e2019GL086729,  https://doi.org/10.1029/2019GL086729

8. Shu, Q., Q. Wang, Z. Song, F. Qiao, J. Zhao, M. Chu and X. Li (2020). Assessment of sea ice extent in CMIP6 with comparison to observations and CMIP5. Geophysical Research Letters, 47, e2020GL087965,  https://doi.org/10.1029/2020GL087965

9. SIMIP Community (2020). Arctic sea ice in CMIP6. Geophysical Research Letters, 47, e2019GL086749,  https://doi.org/10.1029/2019GL086749

10. Watts, M., W. Maslowski, Y. J. Lee, J. C. Kinney, and R. Osinski (2021). A Spatial Evaluation of Arctic Sea Ice and Regional Limitations in CMIP6 Historical Simulations. J. Climate, 34, 6399–6420,  https://doi.org/10.1175/JCLI-D-20-0491.1


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments
---
Production date: 2024-05-31

Produced by: Timothy Williams, Nansen Environmental and Remote Sensing Center (NERSC)


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Quality assessment statement
---
These are the key outcomes of this assessment

- On average the Arctic sea ice thickness (SIT) from the CMIP6 ensemble is fairly unbiased against the altimeter-based estimates, with the exception of October. However the bias starts to become negative towards the end of the historical experiment in 2013 and 2014.

- There are some spatial differences between the CMIP6 models and their ensemble mean. This is shown by the RMSE staying relatively constant throughout the whole evaluation period despite the low bias, indicating that the spatial biases are cancelling each other out. It is also shown by spatial maps of climatologies of the bias, which show that in the ENVISAT period the Beaufort Sea has a strong positive bias, while the Laptev, Kara and Barents Seas have more of a negative bias. In the CryoSat-2 period, the spatial distribution of the bias changes with the Beafort, Laptev, Kara and Barents Seas all have a positive bias, while the Greenland Sea, and the area extending from the north of Greenland to the north pole have a strong negative bias, making the mean bias negative.

- We also compared maps of the climatology of the ensemble mean of the CMIP6 models during the CryoSat-2 period (2010-2020) to the climatology of the ensemble mean in the ENVISAT period (2002-2010). Spatially, the CMIP6 model mean has a fairly uniform drop in thickness when we move from the earlier satellite period to the later one, while the difference in the altimeter climatologies is more variable spatially.

- Other authors have also looked at evaluating CMIP6 models sea ice thickness or volume. [West & Blockley (2024)](https://doi.org/10.5194/gmd-2024-121) compared a subset of the models (those that output heat flux diagnostics) to ice mass-balance buoys, finding that the CMIP6 models they looked at overestimated heat fluxes and thus sea ice melt and growth. A number of authors ([Davy & Outten, 2020](https://doi.org/10.1175/JCLI-D-19-0990.1);  [Xu et al. , 2023](https://doi.org/10.3389/fmars.2023.1223772); [Watts et al. 2021](https://doi.org/10.1175/JCLI-D-20-0491.1)) found that the CMIP6 ensemble mean was similar to the PIOMAS (Pan-Arctic Ice-Ocean Modeling and Assimilation System) reanalysis, although the spread was quite large. [Davy & Outten (2020)](https://doi.org/10.1175/JCLI-D-19-0990.1) also included CMIP5 models in their comparison, noting that both the bias and spread have improved in CMIP6. The spatial patterns in the differences between altimeter and the CMIP6 ensemble mean that we found have also been found by others ([Watts et al., 2021](https://doi.org/10.1175/JCLI-D-20-0491.1); [Henke et al (2023)](https://doi.org/10.1080/15230430.2023.2271592)). In addition, [Kuang et al (2024)](https://doi.org/10.1016/j.accre.2024.06.008) calculated sea ice volume fluxes through six Arctic gateways (the most important being the Fram Strait), finding the multi-model mean also gave a reasonable representation of this flux.

- A number of authors have also used CMIP6 sea ice thickness and volume when considering projections of sea ice. An interesting example of this is the article by [Zhou et al (2022)](https://doi.org/10.1088/1748-9326/ac9d4d), who found that subsetting the CMIP6 models according to how two thickness-based quantities (mean April SIT and the response of the minimum sea ice area to the mean April SIT) compared to the PIOMAS values substantially reduced the spread in projected sea ice extent and when the Arctic Ocean might become ice-free (depending on the warming scenario).
```


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Methodology
---
For our ensemble we include each model in the CMIP6 historical experiment that outputs sea ice thickness, although users should note that choosing a different subset of models could give different results.
Then, for each model in our ensemble, we calculate time series of bias and RMSE for the models compared to sea ice thickness estimates from altimetry - from the ENVISAT product between 2002 and April 2010, and from the CryoSat-2 product after October 2010. We then make plots of the time series showing the median and interquartile limits of the CMIP6 ensemble. There is a separate notebook which looks at these satellite products and addresses whether temporal trends can be seen in these data themselves.

We also calculate monthly thickness climatologies for the ENVISAT period and the CryoSat-2 period, and plot them to compare the spatial distribution of the thickness fields and the biases.

Finally we calculate the difference between the climatologies for the CryoSat-2 and ENVISAT periods, to compare the change in thickness over time for the model and the satellites.

The "Analysis and results" section is organised as follows:

**[](section-1)**

**[](section-2)**

**[](section-3)**

        **[](section-3.1)**

        Plot time series of the error metrics.

        **[](section-3.2)**

        Plot maps of climatologies and their biases for each winter month (October-April).

        **[](section-3.3)**

        Compare the change in thickness moving from the ENVISAT period to the CryoSat-2 period for both the satellites and the CMIP6 ensemble mean.


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.2 Set parameters
---
- Set the time period to be analysed with `year_start` and `year_stop`.
- Set the models to be evaluated with the list `models`.

Choose CMIP6 historical models
"taiesm1", # very high values


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.4 Functions to create the time series
---
These functions are all applied to a single CMIP6 model. They are all applied to the downloaded data before caching.
- `get_satellite_data` downloads the monthly satellite data.
- `regrid` interpolates the model to the satellite grid.
- `get_monthly_interpolated_data` is applied to both the model and satellite data, to take the monthly mean (the satellite data is daily, but this also forces all the models to have the same time coordinate) and interpolate to the satellite grid (only for model data). It also calculates the RMS error (only for satellite data).
- `compare_model_vs_satellite` compares the sea ice thickness from a single CMIP6 model with satellite estimates, calculating error metrics like bias and RMSE in thickness.
- `compute_sea_ice_thickness_diagnostics` downloads the satellite and model data and calls `compare_model_vs_satellite` to get a time series of error metrics.

Remove nan columns
Homogenize time
Get variables
Compute useful variables
Compute output


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.5 Function to plot error time series
---
- `plot_error_time_series` plots the spread of the model errors (compared to the observed sea ice concentration), and also the root mean square observation error for comparison.

da = da.resample(time="1MS").mean()
mark approximate time when CryoSat-2 becomes available


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.6 Functions to create climatologies for plotting
---
- `regridded_monthly_weighted_mean` gets monthly climatologies and then regrids them to the satellite grid. This is applied to the downloaded data before caching.
- `postprocess_map_datasets` is applied after loading the cached data to rename some variables and dimensions in order to make them easier to use and plot.

get monthly climatology
get satellite grid
interpolate the model data onto the satellite grid
reorder months


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 1. Import libraries, set parameters and definitions of functions > 1.7 Functions to plot climatologies
---
- `plot_maps` plots a general climatology.
- `plot_sithick_maps` plots thickness climatologies.
- `plot_bias_maps` plots differences between the climatologies of the satellites and the CMIP6 ensemble mean.
- `plot_change_maps` plots the difference between a climatology for the Cryosat-2 period (2010-2014) and the Envisat period (2002-2010).

(section-2)=


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 2. Download and transform the data > 2.2 Create the climatologies
---
We first save monthly climatologies for the satellite data.
Then we loop over all the models we are considering and apply `regridded_monthly_weighted_mean` to create climatologies, before taking their ensemble mean.

(section-3)=


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 3. Results > 3.1 Time series of evaluation metrics
---
Below we plot time series of the bias and RMSE in the sea ice thickness of the CMIP6 models compared to the altimetry data.
We have taken an ensemble of models (all those that output sea ice thickness), interpolated them to the altimeter grid, and calculated time series of area-weighted error metrics for each model. We have then plotted the median and the inter-quartile limits for the ensemble for each metric. Also note that the altimeter data is only available in the winter months (October - April), which is why the time series have gaps. The vertical red lines (mid-2010) divide the ENVISAT period (2002-2010) from the CryoSat-2 one (2010-2020 for the CDR, but the historical CMIP6 experiments finish in 2014).

The bias in the models are generally about the same level as the observation error level ($\pm\sigma_{\rm obs}$, where $\sigma_{\rm obs}$ is the RMS error in the observations), and the ensemble median is close to zero for most of the time interval considered. Around 2014, however, the bias drops to about -0.25m compared to CryoSat-2.

Within each winter the bias is generally highest in October -  about 0.3m - 0.5m for ENVISAT and also in the 2011 winter. The intra-winter variability in the bias seems to reduce after this however.

It is worth remembering that ENVISAT has its own bias which has been estimated to be about -0.6m so the models are probably also underestimating the thickness by that amount in the ENVISAT period [(CDS SIT Quality Assessment Report)](https://dast.copernicus-climate.eu/documents/satellite-sea-ice-thickness/level-3/v3-0/WP2-FDDP-2022-09_C3S2-Lot3_PQAR-of-v3.0-SeaIceThickness-products_v3.2_Final.pdf). The bias in CryoSat-2 has been estimated to be about -0.11m [(CDS SIT Quality Assessment Report)](https://dast.copernicus-climate.eu/documents/satellite-sea-ice-thickness/level-3/v3-0/WP2-FDDP-2022-09_C3S2-Lot3_PQAR-of-v3.0-SeaIceThickness-products_v3.2_Final.pdf) so it is a little surprising that the model bias does not become negative immediately upon the change in satellite but instead stays around zero from 2010 to 2013. However, the validation data which was used to estimate the bias was limited in space to 3 or 4 drilling sites in the Beafort Sea and some airborne electromagnetic tracks in the Beafort Sea and north of Greenland [(CDS SIT Quality Assessment Report)](https://dast.copernicus-climate.eu/documents/satellite-sea-ice-thickness/level-3/v3-0/WP2-FDDP-2022-09_C3S2-Lot3_PQAR-of-v3.0-SeaIceThickness-products_v3.2_Final.pdf), so the satellite biases may not apply to the whole Arctic.

The RMSE is also relatively constant over the entire period and can be about 50% above the plotted error range which is taken to be $(\sigma_{\rm obs}, \sqrt{2}\sigma_{\rm obs})$. (This approximate error level comes from considering uncorrelated errors in the model and the observation time series when going through the calculation of the MSE - then the MSE is comparable to the error level if it is around $\sigma_{\rm obs}^2 + \sigma_{\rm mod}^2$, which becomes $2\sigma_{\rm obs}^2$ if we assume the model and observation errors are similar.) However in some years the median is near the error range and there is decent overlap between the IQL from the models and the error range. There is not such a clear intra-winter variability in this statistic as in the bias.

(section-3.2)=


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 3. Results > 3.2 Spatial distribution of errors
---
Below we plot monthly climatologies calculated from 2002 to 2010, of the thicknesses from ENVISAT and CMIP6 (the ensemble mean over all models), and the bias (model - observations) in the CMIP6 mean. The ice in the Canadian Archipelago is consistently very thick in the models for all months. In October there is a strong positive bias in the Beaufort sea, which, while present in all other months, is slightly reduced. This bias also extends spatially into the Siberian Sea for all months, although it is not as strong there. In the Eastern Arctic (the Laptev, Kara and Barents Seas), the Labrador Sea and Hudson Bay there are consistent negative biases. In the Greenland Sea the thickness is mostly negatively biased but in October there is a reasonably sized region of positive bias as well. In other months there is some positive bias there as well but the regions are smaller. Bear in mind, however, that considering such a dynamic region on a monthly time-scale may limit the conclusions that can be made about the Greenland Sea.

Below we plot monthly climatologies calculated from 2010 to 2020, of the thicknesses from CryoSat-2 and CMIP6 (the ensemble mean over all models), and the bias (model - observations) in the CMIP6 mean. The ice in the Canadian Archipelago is still very thick in the models for all months. In October, there is a region of positive bias in the Beaufort, Siberian and Laptev seas. The extent of this region decreases as the months progress. There is a region of negative bias to the north of Greenland and the Canadian Archipelago, since the thicker ice off these coasts extends to the north pole in CryoSat-2 but is quite localised at the coasts in the models.

The ice in the Hudson Bay has a moderate negative bias, while the bias in the northern part of the Labrador Sea grows from around zero in November to about 25-50cm in April. Further south in the Labrador Sea there is a slight negative bias.

(section-3.3)=


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > Analysis and results > 3. Results > 3.3 Change in thickness over time
---
Below we compare the differences (later - earlier) in the two climatologies for the ENVISAT period (2002-2010) and the overlap between the CryoSat-2 period and the historical experiment (2010-2024). The satellite products show a general decrease, with the exception of the Beaufort Sea in October, while the CMIP6 models show a near-uniform drop between the two periods. The magnitude of the drop is higher for the satellites however.
If we follow the estimated biases for each satellite, the change in the satellite thickness should drop by about 40cm, since ENVISAT has a larger negative bias than CryoSat-2. Hence we can theoretically trust decreases in the satellite thickness more than increases. However, as mentioned earlier, the validation data which was used to estimate the satellite biases was quite limited spatially [(CDS SIT Quality Assessment Report)](https://dast.copernicus-climate.eu/documents/satellite-sea-ice-thickness/level-3/v3-0/WP2-FDDP-2022-09_C3S2-Lot3_PQAR-of-v3.0-SeaIceThickness-products_v3.2_Final.pdf).

The short length of the time periods makes it difficult to see trends, which is also the conclusion of the notebook about the satellite sea ice thickness data, and the advice in the [CDS SIT Quality Assessment Report](https://dast.copernicus-climate.eu/documents/satellite-sea-ice-thickness/level-3/v3-0/WP2-FDDP-2022-09_C3S2-Lot3_PQAR-of-v3.0-SeaIceThickness-products_v3.2_Final.pdf). This is also why many authors use the PIOMAS reanalysis as a reference sea ice thickness product. [Davy & Outten (2020)](https://doi.org/10.1175/JCLI-D-19-0990.1) found that the CMIP6 ensemble mean had a similar trend to PIOMAS, showing thinning of sea ice and calculated over longer time intervals.


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > ℹ If you want to know more > Key resources
---
Introductory sea ice materials:
- [Role of sea ice in the climate](https://marine.copernicus.eu/explainers/why-ocean-important/sea-ice)
- [Sea ice as an indicator of climate change](https://climate.copernicus.eu/climate-indicators/sea-ice#)
- [Observing sea ice with satellites](https://www.metoffice.gov.uk/research/climate/cryosphere-oceans/sea-ice/measure)

Introductory CMIP6 materials:
- [A short introduction to CMIP and CMIP6](https://www.wcrp-climate.org/wgcm-cmip/cmip-video)
- [CMIP6: the next generation of climate models explained](https://www.carbonbrief.org/cmip6-the-next-generation-of-climate-models-explained/)

Code libraries used:
* [C3S EQC custom functions](https://github.com/bopen/c3s-eqc-automatic-quality-control/tree/main/c3s_eqc_automatic_quality_control), `c3s_eqc_automatic_quality_control`,  prepared by [B-Open](https://www.bopen.eu/)


EQC Quality Assessment: "Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments"
Dataset: projections-cmip6 [CDS]
Aspect: model-performance_q03  |  Category: Climate_Projections
Section: Evaluation of the Arctic sea ice thickness in the CMIP6 historical experiments > ℹ If you want to know more > References
---
1. [CDS Satellite Sea Ice Thickness Product Quality Assessment Report](https://dast.copernicus-climate.eu/documents/satellite-sea-ice-thickness/level-3/v3-0/WP2-FDDP-2022-09_C3S2-Lot3_PQAR-of-v3.0-SeaIceThickness-products_v3.2_Final.pdf)

1. Davy, R., and S. Outten (2020). The Arctic Surface Climate in CMIP6: Status and Developments since CMIP5. J. Climate, 33, 8047–8068,  https://doi.org/10.1175/JCLI-D-19-0990.1

1. Henke, M., F. Cassalho, T. Miesse, C. M. Ferreira, J. Zhang and T. M. Ravens (2023). Assessment of Arctic sea ice and surface climate conditions in nine CMIP6 climate models. Arctic, Antarctic, and Alpine Research, 55(1),  https://doi.org/10.1080/15230430.2023.2271592

1. Kuang, H. Y., Sun, S. Z., Ye, Y. F., Wang, S. Y., Bi, H. B., Chen, Z. Q., & Cheng, X. (2024). An assessment of the CMIP6 performance in simulating Arctic sea ice volume flux via Fram Strait. Advances in Climate Change Research, 15(4), 584-595,  https://doi.org/10.1016/j.accre.2024.06.008

1. Watts, M., W. Maslowski, Y. J. Lee, J. C. Kinney, and R. Osinski (2021). A Spatial Evaluation of Arctic Sea Ice and Regional Limitations in CMIP6 Historical Simulations. J. Climate, 34, 6399–6420,  https://doi.org/10.1175/JCLI-D-20-0491.1

1. West, A. E., & Blockley, E. W. (2024). CMIP6 models overestimate sea ice melt, growth & conduction relative to ice mass balance buoy estimates. Geoscientific Model Development Discussions, 2024, 1-33,  https://doi.org/10.5194/gmd-2024-121

1. Xu, M., & Li, J. (2023). Assessment of sea ice thickness simulations in the CMIP6 models with CICE components. Frontiers in Marine Science, 10, 1223772,  https://doi.org/10.3389/fmars.2023.1223772

1. Zhou, X., Wang, B., & Huang, F. (2022). Evaluating sea ice thickness simulation is critical for projecting a summer ice-free Arctic Ocean. Environmental Research Letters, 17(11), 114033,  https://doi.org/10.1088/1748-9326/ac9d4d


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability
---
Production date: 19-01-2025

Produced by: CMCC foundation - Euro-Mediterranean Center on Climate Change. Albert Martinez Boti.


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Quality assessment question
---
* **How well do CMIP6 projections represent historic inter-annual variability of precipitation across the Indian sub-continent? Are there trends affecting precipitation inter-annual variability?**

Most research has focused on projected long-term changes in mean climate and extremes, while changes in variability have received less attention. However, understanding the evolution of precipitation variability is crucial for accurately modelling climate phenomena and quantify climate change impacts, particularly on water resources and related socioeconomic factors [[1]](https://doi.org/10.1111/gcb.12581),[[2]](https://doi.org/10.1016/j.wace.2020.100277). This notebook evaluates the capability of a subset of 16 [CMIP6](https://cds.climate.copernicus.eu/datasets/projections-cmip6?tab=overview) Global Climate Models (GCMs) to represent inter-annual variability in seasonal mean precipitation over the Indian subcontinent, employing variance and the coefficient of variation (standard deviation divided by the mean) as measures of variability.

Systematic errors are assessed by comparing model outputs against the [ERA5](https://cds.climate.copernicus.eu/datasets/reanalysis-era5-complete?tab=doc) reanalysis, which serves as the reference dataset. The analysis focuses on JJAS (June, July, August and September), traditionally regarded as the Indian Summer Monsoon period, though the methodology is adaptable for annual or other seasonal analyses. Spatial patterns of the variance and the coefficient of variation are examined for JJAS mean precipitation over the historical period from 1940 to 2014. Additionally, the temporal evolution of the spatially-averaged variance and coefficient of variation, including trends, is analysed using a 30-year moving window and presented as time series.

The analyses presented in this notebook provide valuable insights into the reliability of a subset of CMIP6 models in capturing precipitation inter-annual variability. Understanding the performance of these models can help the water management sector to make more informed decisions and supports the use of CMIP6 Global Climate Models (GCMs) in designing strategies to adapt to projected changes in variability.


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Quality assessment statement
---
These are the key outcomes of this assessment

* The choice of variability formulation is crucial when describing inter-annual variability during the summer Monsoon. The two formulations considered within this assessment reveal distinct spatial patterns.

* Variance analyses reveal model-dependent differences in bias magnitude, sign, and spatial distribution. Despite these variations, all models show strong links to orography and high-precipitation regions, where biases remain significant due to high precipitation, even though they are not necessarily large relative to the mean. The coefficient of variation removes this effect by normalising the standard deviation (i.e., the square root of the variance) by the mean.

* Model differences are less pronounced for the coefficient of variation than for variance. Most models underestimate it in the inland north-west, where seasonal mean precipitation is very low or near zero.

* The 30-year time series of spatially averaged variance and coefficient of variation show a decreasing trend for ERA5, suggesting a decline in inter-annual variability over the historical period. This is evident from both variance magnitude—potentially influenced by the mean's decline [[3]](https://doi.org/10.1007/978-981-15-4327-2_3)—and the coefficient of variation, which removes the effect of the mean. However, the CMIP6 ensemble median for the subset of considered models shows the opposite trend.

* Understanding and correcting the biases of CMIP6 projections of precipitation variability may increase confidence when analysing projections assessing the future inter-annual variability which can help on planning for water resource allocation, flood mitigation, and agricultural management, ensuring resilience against the challenges posed by a changing climate.

```

attachment:a920aca4-0f57-49cc-9f87-296209edb822.png
---
width: 1200px
alt: Mean Bias variance
---
Fig A. i. Variance of JJAS mean precipitation over the historical period (1940–2014). The layout includes data corresponding to: (a) ERA5, (b) the ensemble median (defined as the median of the variance values for the selected subset of models, calculated for each grid cell), (c) the bias of the ensemble median, and (d) the ensemble spread (calculated as the standard deviation of the variance values for the selected subset of models). Fig A.ii. Same as Fig. A.i. but for the coefficient of variation.
```


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Methodology
---
A subset of 16 models from the [CMIP6](https://cds.climate.copernicus.eu/datasets/projections-cmip6?tab=overview) project is used to calculate the variance and the coefficient of variation (standard deviation divided by the mean), which serve as formulations to characterise precipitation inter-annual variability in this notebook. Spatial patterns of variance and the coefficient of variation for seasonal mean precipitation, along with their associated biases (using [ERA5](https://cds.climate.copernicus.eu/datasets/reanalysis-era5-complete?tab=doc) as reference), are displayed for each model and the ensemble median (per grid cell). Additionally, the spatially averaged temporal evolution of the variance and coefficient of variation, including trends, is analysed using a 30-year moving window and presented as time series, following a methodology similar to [[4]](https://doi.org/10.1007/s00704-022-03972-2).

The analysis focuses on the Indian subcontinent during the historical period from 1940 to 2014 and examines the JJAS period, traditionally regarded as the Indian Summer Monsoon (or rainy) season, although the methodology can be adapted for other seasonal or annual periods.

The analysis and results follow the next outline:

**[](climate_projections-cmip6_climate-and-weather-extremes_q01:section-1)**
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.1)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.2)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.3)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.4)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.5)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.6)

**[](climate_projections-cmip6_climate-and-weather-extremes_q01:section-2)**
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-2.1)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-2.2)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-2.3)

**[](climate_projections-cmip6_climate-and-weather-extremes_q01:section-3)**
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.1)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.2)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.3)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.4)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.5)
 * [](climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.6)


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 1. Parameters, requests and functions definition > 1.2. Define Parameters
---
In the "Define Parameters" section, various customisable options for the notebook are specified:

- The initial and ending year used for the historical period can be specified by changing the parameters `year_start` and `year_stop` (1940-2014 is chosen).
- The `timeseries` set the temporal period. For instance, selecting "JJAS" implies considering only the June, July, August, September season.
- `area` allows specifying the geographical domain of interest. A domain including the Indian sub-continent has been selected.
- The `interpolation_method` parameter allows selecting the interpolation method when regridding is performed.
- The `chunk` selection allows the user to define if dividing into chunks when downloading the data on their local machine. Although it does not significantly affect the analysis, it is recommended to keep the default value for optimal performance.
- `variable` and `collection_id` are not customisable for this assessment and are set to 'precipitation' and 'CMIP6'. Expert users can use this notebook as a guide to create similar analyses for other variables or model sets (such as CORDEX).

Choose annual or seasonal timeseries
Interpolation method
Area to show
Chunks for download
Collection id

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.3)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 1. Parameters, requests and functions definition > 1.3. Define models
---
The following climate analyses are performed considering a subset of GCMs from CMIP6.

The selected CMIP6 models have available both the historical and SSP5-8.5 experiments.

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.4)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 1. Parameters, requests and functions definition > 1.4. Define ERA5 request
---
Within this notebook, ERA5 serves as the reference product. In this section, we set the required parameters for the cds-api data-request of ERA5.

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.5)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 1. Parameters, requests and functions definition > 1.5. Define model requests
---
In this section we set the required parameters for the cds-api data-request.

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-1.6)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 1. Parameters, requests and functions definition > 1.6. Functions to cache
---
In this section, functions that will be executed in the caching phase are defined. Caching is the process of storing copies of files in a temporary storage location, so that they can be accessed more quickly. This process also checks if the user has already downloaded a file, avoiding redundant downloads.

Functions description:

- `get_grid_out` and `add_bounds` ensure the regrid is performed properly.

- The `select_timeseries` function subsets the dataset based on the chosen `timeseries` parameter.

- The `compute_rolling_variance` function calculates the rolling variance and coefficient of variation using a 30-year window (which are needed when displaying the time series afterwards). It also computes the variance and coefficient of variation for the entire period.

- The `compute_interannual_variance` function selects the season using the `select_timeseries` function. It then computes the rolling variance (and coefficient of variation) over the historical period (1940–2014) by calling `compute_rolling_variance`. The window width is 30 years, with a step size of one year. The variance and coefficient of variation is also calculated for the whole historical period.

Select the time series
Get seasonal/annual means
Change units
Compute variances
Handle grid output if requested

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-2)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 2. Downloading and processing > 2.1. Download and transform ERA5
---
In this section, the `download.download_and_transform` function from the 'c3s_eqc_automatic_quality_control' package is used to download ERA5 reference monthly data, select the season (e.g., "JJAS" in this case) and compute the variance and coefficient of variation for the entire historical period, as well as for a 30-year rolling window. The results are then cached to prevent redundant downloads and processing.

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-2.2)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 2. Downloading and processing > 2.2. Download and transform models
---
In this section, the `download.download_and_transform` function from the 'c3s_eqc_automatic_quality_control' package is employed to download daily data from the CMIP6 models, select the season ("JJAS" in this example), compute the variance and coefficient of variation for the entire historical period, as well as for a 30-year rolling windows, interpolate to the ERA5 grid (only when specified; otherwise, the original model grid is maintained), and cache the results to avoid redundant downloads and processing.

Original model
Interpolated model

```text
model='access_cm2'
model='bcc_csm2_mr'
model='cmcc_esm2'
model='cnrm_cm6_1_hr'
model='cnrm_esm2_1'
model='ec_earth3_cc'
model='gfdl_esm4'
model='inm_cm4_8'
model='inm_cm5_0'
model='mpi_esm1_2_lr'
model='miroc6'
model='miroc_es2l'
model='mri_esm2_0'
model='noresm2_mm'
model='nesm3'
model='ukesm1_0_ll'
```

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-2.3)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 2. Downloading and processing > 2.3. Change some attributes
---
Drop 'bnds' dimension if it exists for each dataset and for the interpolated dataset

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-3)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results
---
This section will display the following results:

- Maps showing the spatial distribution of the **variance of JJAS mean precipitation** calculated for the whole historical period 1940-2014 comparing ERA5 and the ensemble median (defined as the median of the variance values for the selected subset of models, calculated for each grid cell). The layout include ERA5, the ensemble median, the ensemble median bias and the ensemble spread (derived as the standard deviation of the variance for the selected subset of models). The same exact layout of maps is displayed for the coefficient of variation.

- Maps showing the spatial distribution of the **variance of JJAS mean precipitation** calculated for the whole historical period 1940-2014 for each model. The same exact layout of maps is displayed for the coefficient of variation.

- **Bias maps of the variance and coefficient of variation of JJAS mean precipitation** calculated for the whole historical period 1940–2014 for each model.

- **Timeseries** showing the evolution of the 30-year variance and coefficient of variation (standard deviation divided by the mean) of JJAS mean precipitation over the historical period, including trends and inter-model spread.

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.1)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results > 3.1. Define plotting functions
---
The functions presented here are used to plot layouts of the variance and coefficient of variation of seasonal mean precipitation over the whole historical period.

Three layout types can be displayed, depending on the chosen function:

1. Layout including the reference ERA5 product, the ensemble median, the bias of the ensemble median, and the ensemble spread: `plot_ensemble()` is used.
2. Layout including every model: `plot_models()` is employed.
3. Layout including the bias of every model: `plot_models()` is used again.

Define function to plot the caption of the figures (for the ensemble case)
Add each line to the figure
end captioning
Define function to plot the caption of the figures (for the individual models case)
Add caption to the figure
Add each line to the figure

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.2)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results > 3.2. Plot ensemble maps
---
In this section, we invoke the `plot_ensemble()` function to visualise the variance values for JJAS mean precipitation over the period 1940–2014 for: (a) the reference ERA5 product, (b) the ensemble median (defined as the median of the variance values for the selected subset of models, calculated for each grid cell), (c) the bias of the ensemble median, and (d) the ensemble spread (calculated as the standard deviation of the variance for the selected subset of models). The same exact layout of maps is displayed for the coefficient of variation.

Change default colorbars
Fig number counter
Common title
get the variance of the whole period
Show the plot
Increment figure number

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.3)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results > 3.3. Plot model maps
---
In this section, we invoke the `plot_models()` function to visualise the variance and the coefficient of variation of JJAS mean precipitation over the period 1940–2014 for every model individually. Note that the model data used in this section maintains its original grid.

Select the mean values of the variances
Show the plot
Increment figure number

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.4)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results > 3.4. Plot bias maps
---
In this section, we invoke the `plot_models()` function to visualise the bias of the variance and the coefficient of variation of JJAS mean precipitation over the period 1940–2014 for every model individually. Note that the model data used in this section has previously been interpolated to the ERA5 grid.

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.5)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results > 3.5. Timeseries of the coefficient of variation
---
This section examines the time series of spatially averaged variance and coefficient of variation, calculated from JJAS mean precipitation over the historical period using a 30-year window. The analysis compares the CMIP6 ensemble median, ERA5, and individual models, highlighting trends for both the CMIP6 ensemble median and ERA5. It also displays the ensemble spread, along with the slope values of the trends for ERA5 and the CMIP6 ensemble median.

Define the colors
Cutout region
Get the spatial mean for ERA5
Get the spatial mean for CMIP6, preserving each model
Calculate the median, ensemble mean, and standard deviation
Initialize list to store trend information
Plot individual model series for each variance

Fig 7. Time series of the spatially averaged values for the chosen variability formulations: (a) variance and (b) coefficient of variation for the JJAS mean precipitation over the period 1940–2014. The variance and coefficient of variation have been calculated using a 30-year window. Each time series displays the CMIP6 ensemble median (green), ERA5 (black), and individual models (grey), with dashed lines indicating the trend for the CMIP6 ensemble median and ERA5. The shaded area represents the ensemble spread. A grey box at the bottom displays the trend slope.

(climate_projections-cmip6_climate-and-weather-extremes_q01:section-3.6)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results > 3.6. Results summary and discussion
---
* Substantial differences in spatial patterns can be observed depending on the formulation used to represent inter-annual variability during the Monsoon season (JJAS). This notebook uses two formulations: variance and the coefficient of variation (standard deviation divided by the mean).

* Large differences in sign, magnitude, and spatial patterns of the biase are observed in the analyses using variance to represent inter-annual variability. Despite the differences across the subset of considered CMIP6 models, there is agreement in showing strong dependencies on orography and regions with high precipitation. These regions exhibit high biases due to elevated mean JJAS precipitation, although these biases are not necessarily large relative to the mean. Using the coefficient of variation eliminates this effect and may be more useful for assessing variability normalised to the mean amount of precipitation during the Monsoon period.

* Generally, there are smaller differences depending on the model selection for the coefficient of variation, compared to the differences observed for the variance formulation. Inland north-western regions show large negative biases. These regions have, on average, near-zero or very low seasonal averages. One hypothesis to explain this may be that unusually wet seasons are underestimated in comparison to ERA5.

* The time series displaying the evolution of variance and coefficient of variation (using a 30-year window) show a decline for ERA5 in the considered region. Based on the calculated trend, an approximate 50% reduction is estimated when comparing the start and end of the period for the variance formulation. This decrease is smaller when considering the coefficient of variation (11%). The ensemble median trend for the subset of considered CMIP6 models shows an increase of around 7% for the variance and 6% for the coefficient of variation over the same period, which differs from the trends observed in ERA5. It is important to note that the region studied is extensive. The time series represent spatially averaged values, and their results should be interpreted with care, as focusing on sub-regions within the domain may yield significantly different outcomes. Additionally, CMIP6 models exhibit notable inter-model spread, and a larger ensemble could also influence the results.

* The time series results suggest that year-to-year variability in monsoon precipitation has become less pronounced according to ERA5. This is evident not only from the variance magnitude, which can be influenced by the evolution of the mean (shown to decrease during this period [[3]](https://doi.org/10.1007/978-981-15-4327-2_3)), but also from the coefficient of variation, which removes this dependency by normalising the standard deviation with the mean.

* What do the results mean for users?
    * **Importance of the selection of the variability formulation**. The coefficient of variation has less dependency on orography because it is normalised by the mean. Moreover, the differences across the considered subset of models in the spatial pattern of the bias for the coefficient of variation appear to be smaller than in the case of variance. However, special attention needs to be given when using the coefficient of variation for the inland north-western region (where the seasonal mean precipitation is usually very low or near zero).
    * **ERA5 has biases** [[5]](https://doi.org/10.1002/qj.3616)[[6]](https://doi.org/10.3390/atmos12111462). This assessment uses it because a longer period is available and allows a more robust analysis of the temporal evolution of the variability. However the user may **consider using other reference climate products such as the [GPCP](https://cds.climate.copernicus.eu/datasets/satellite-precipitation?tab=overview)**. This could lead to different results.
    * ERA5 shows a **decrease in inter-annual variability over the historical period for the considered region**, while the CMIP6 ensemble median indicates a slight increase. It is **important to consider a larger ensemble and note that the time series in this assessment represent spatially averaged values**, which may yield different results when focusing on other sub-regions.
    * Given the challenges in replicating the spatial patterns for variance and the coefficient of variation, as well as the difficulties in representing its temporal evolution, **bias correction of CMIP6 projections may be essential for effectively using future projections of inter-annual precipitation variability. This enhancement could encourage the water management sector to adopt CMIP6 future projections, which can be extremely useful when designing strategies to adapt to projected changes in precipitation.**
    * **Assessments using regional climate models (RCMs) may produce different results** (not necessarily better) due to their higher spatial resolution, better representation of local processes, and more detailed treatment of orography and land-atmosphere interactions.


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > Analysis and results > 3. Plot and describe results > 3.6. Results summary and discussion
---
Assessments using regional climate models (RCMs) may produce different results** (not necessarily better) due to their higher spatial resolution, better representation of local processes, and more detailed treatment of orography and land-atmosphere interactions.

RESULTS NOTE:

It is important to note that the results presented are specific to the 16 models chosen, and users should aim to assess as wide a range of models as possible before making a sub-selection.


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > ℹ If you want to know more > Key resources
---
Some key resources and further reading were linked throughout this assessment.

The CDS catalogue entries for the data used were:
* CMIP6 climate projections (Monthly - Precipitation): https://cds.climate.copernicus.eu/datasets/projections-cmip6?tab=overview
* ERA5 monthly averaged data on single levels from 1940 to present (mean total precipitation rate): https://cds.climate.copernicus.eu/datasets/reanalysis-era5-single-levels-monthly-means?tab=overview

Code libraries used:
* [C3S EQC custom functions](https://github.com/bopen/c3s-eqc-automatic-quality-control/tree/main/c3s_eqc_automatic_quality_control), `c3s_eqc_automatic_quality_control`,  prepared by [B-Open](https://www.bopen.eu/)


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q10  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation inter-annual variability > ℹ If you want to know more > References
---
[[1]](https://doi.org/10.1111/gcb.12581) Thornton, P. K., Ericksen, P. J., Herrero, M., and Challinor, A. J., 2014. Climate variability and vulnerability to climate change: A review. Global Change Biol. 20, https://doi.org/10.1111/gcb.12581

[[2]](https://doi.org/10.1016/j.wace.2020.100277) Masroor, M., Rehman, S., Avtar, R., Sahana, M., Ahmed, R., Sajjad, H., 2020. Exploring climate variability and its impact on drought occurrence: evidence from Godavari Middle sub-basin, India. Weather and Climate Extremes. 30, 100277. https://doi.org/10.1016/j.wace.2020.100277

[[3]](https://doi.org/10.1007/978-981-15-4327-2_3) Kulkarni, A. et al., 2020. Precipitation Changes in India. In: Krishnan, R., Sanjay, J., Gnanaseelan, C., Mujumdar, M., Kulkarni, A., Chakraborty, S. (eds) Assessment of Climate Change over the Indian Region. Springer, Singapore. https://doi.org/10.1007/978-981-15-4327-2_3

[[4]](https://doi.org/10.1007/s00704-022-03972-2) Longobardi, A., Boulariah, O., 2022. Long-term regional changes in inter-annual precipitation variability in the Campania Region, Southern Italy. Theor Appl Climatol 148, 869–879. https://doi.org/10.1007/s00704-022-03972-2

[[5]](https://doi.org/10.1002/qj.3616) Ramon, J., Lledó L., Torralba, V., Soret, A., Doblas-Reyes, F.J., 2019. What global reanalysis best represents near-surface winds?. Q J R Meteorol Soc. 2019; 145: 3236–3251. https://doi.org/10.1002/qj.3616

[[6]](https://doi.org/10.3390/atmos12111462) Hassler, B., and Lauer, A., 2021. Comparison of Reanalysis and Observational Precipitation Datasets Including ERA5 and WFDE5. Atmosphere, 12, 1462. https://doi.org/10.3390/atmos12111462


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability
---
Production date: 31-03-2025

Produced by: CMCC foundation - Euro-Mediterranean Center on Climate Change. Albert Martinez Boti.


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Quality assessment question
---
* **How well do CMIP6 projections represent historical intra-seasonal variability of Monsoon precipitation across the Indian subcontinent? Are there trends affecting the intra-seasonal variability of Monsoon precipitation?**

Most research has focused on projected long-term changes in mean climate and extremes, while changes in variability have received less attention. However, understanding the evolution of precipitation variability is crucial for accurately modelling climate phenomena and quantifying the impacts of climate change, particularly on water resources and related socioeconomic factors [[1]](https://doi.org/10.1111/gcb.12581), [[2]](https://doi.org/10.1016/j.wace.2020.100277). This notebook evaluates the capability of a subset of 16 [CMIP6](https://cds.climate.copernicus.eu/datasets/projections-cmip6?tab=overview) Global Climate Models (GCMs) to represent monsoon intra-seasonal variability over the Indian subcontinent. Specifically, it evaluates the High-Frequency Intra-Seasonal Oscillations (HFISOs) with a characteristic period of 10–20 days, generally associated with the Quasi-Biweekly Oscillation (QBWO) [[3]](https://doi.org/10.1111/j.2153-3490.1980.tb01717.x), [[4]](https://doi.org/10.1175/2007MWR1991.1), [[5]](https://doi.org/10.1175/2011JCLI3916.1). The Low-Frequency Intra-Seasonal Oscillations (LFISOs) — which have a characteristic period of 30–60 days [[6]](https://doi.org/10.1175/1520-0469%281971%29028%3C0702:DOADOI%3E2.0.CO;2), [[7]](https://doi.org/10.1175/1520-0469%281972%29029<1109:DOGSCC>2.0.CO;2) and are usually linked to the well-known Madden-Julian Oscillation (MJO) mode — are also studied.

Systematic errors are obtained by comparing LFISOs and HFISOs — calculated from model daily outputs — with those from the ERA5 reanalysis, which serves as the reference dataset. These oscillations are extracted by applying a pass-band filter (10–20 days for HFISOs and 30–60 days for LFISOs) to daily precipitation anomalies. To assess the intensity of these oscillations [[8]](https://doi.org/10.1038/s41612-022-00253-7) , variance and the coefficient of variation of the filtered data are calculated for the summer monsoon period (JJAS) from 1940 to 2014. This assessment presents the spatial patterns of the variance and coefficient of variation of LFISOs and HFISOs, as well as their evolution. The latter is evaluated through time series of spatially averaged variance and coefficient of variation of the filtered data using 30-year moving windows.

Biases and future changes in LFISOs and HFISOs are highly relevant to users, as these modes are often associated with: (1) active-break cycles (i.e., alternating ‘active’ phases of heavy rainfall and quiescent phases during the peak monsoon season in July and August [[9]](https://doi.org/10.1002/jgrd.50422), [[10]](https://doi.org/10.1175/JCLI-D-18-0319.1)), (2) the activity of synoptic systems such as monsoon lows (e.g., [[11]](https://doi.org/10.1175/JCLI-D-16-0360.1)) and (3) late monsoon rainfall [[12]](https://doi.org/10.1007/s00382-019-04946-3). This evaluation complements another [assessment](https://ecmwf-projects.github.io/c3s2-eqc-quality-assessment/Climate_Projections/CMIP6/climate_projections-cmip6_validation_q10.html#) that examined the representation of historic inter-annual precipitation variability across the Indian subcontinent.


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Quality assessment statement
---
These are the key outcomes of this assessment

* This assessment complements a previous analysis of [historical inter-annual precipitation variability across the Indian subcontinent](https://ecmwf-projects.github.io/c3s2-eqc-quality-assessment/Climate_Projections/CMIP6/climate_projections-cmip6_validation_q10.html#).

* Intra-seasonal and inter-annual variability assessments show similar spatial patterns, suggesting a connection between the two [[10]](https://doi.org/10.1175/JCLI-D-18-0319.1)

* Variability formulation matters: the two metrics used highlight different spatial features of intra-seasonal variability.

* Variance shows model-dependent biases linked to orography and wet regions. The Coefficient of Variation (CV) normalises for mean precipitation but can lead to artificially high values in extremely dry areas.

* CV differences between models are smaller than those for variance. Most models underestimate CV in dry inland regions.

* HFISOs and LFISOs variance and CV exhibit similar spatial patterns.

* CMIP6 models poorly capture observed trends in HFISO and LFISO variability. ERA5 shows decreasing variance and increasing CV, which are not reproduced by the ensemble median.

*  Bias correction may be necessary before using CMIP6 precipitation projections in water management applications. Similarities in inter-annual and intra-seasonal variability biases suggest corrections may be required at the daily level.

```

attachment:8aac0bed-aa00-4ba0-9343-cbc722fc0ca8.png
---
width: 900px
alt: Mean Bias variance
---
Fig A. i. Variance of HFISOs in daily precipitation anomalies over the historical period (1940–2014). The layout includes data corresponding to: (a) ERA5, (b) the ensemble median (defined as the median of the Variance of HFISOs values for the selected subset of models, calculated for each grid cell), (c) the bias of the ensemble median, and (d) the ensemble spread (calculated as the standard deviation of the Variance of HFISOs values for the selected subset of models). Before calculating the variance, a Lanczos band-pass filter has been applied to the daily precipitation anomalies to isolate the 10–20-day high-frequency intraseasonal oscillations (HFISOs). Fig A.ii. Same as Fig. A.i. but for the coefficient of variation of HFISOs.
```


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Methodology
---
A subset of 16 models from the [CMIP6](https://cds.climate.copernicus.eu/datasets/projections-cmip6?tab=overview) project is used to evaluate the representation of intra-seasonal precipitation variability over the Indian subcontinent. The analysis focuses on two intra-seasonal timescales: High-Frequency Intra-Seasonal Oscillations (HFISOs) with periods of 10–20 days and Low-Frequency Intra-Seasonal Oscillations (LFISOs) with periods of 30–60 days. LFISOs and HFISOs are identified using a pass-band filter (10–20 days for HFISOs and 30–60 days for LFISOs) applied to daily precipitation anomalies, computed relative to the climatological precipitation for each day of the year. The intensity of these intra-seasonal oscillations is assessed by calculating the variance and Coefficient of Variation (CV) of the filtered data, following the approach in [[8]](https://doi.org/10.1038/s41612-022-00253-7), for the summer monsoon period (JJAS) from 1940 to 2014.

The analyses performed within this notebook examine the spatial patterns of variance and the Coefficient of Variation for LFISOs and HFISOs, along with the associated biases for each model and the ensemble median (per grid cell). [ERA5](https://cds.climate.copernicus.eu/datasets/reanalysis-era5-complete?tab=doc) serves as the reference dataset. Note that although HFISOs are generally associated with the Quasi-Biweekly Oscillation (QBWO) [[3]](https://doi.org/10.1111/j.2153-3490.1980.tb01717.x), [[4]](https://doi.org/10.1175/2007MWR1991.1), [[5]](https://doi.org/10.1175/2011JCLI3916.1), and LFISOs are typically linked to the Madden–Julian Oscillation (MJO) [[6]](https://doi.org/10.1175/1520-0469%281971%29028%3C0702:DOADOI%3E2.0.CO;2), [[7]](https://doi.org/10.1175/1520-0469%281972%29029<1109:DOGSCC>2.0.CO;2), the filtering method is subject to limitations, including signal contamination and overlap. Therefore, the variability captured in each band is not exclusively attributable to the QBWO or MJO.

Finally, the temporal evolution of spatially averaged intra-seasonal variability is analysed using time series of variance and the coefficient of variation, computed with a 30-year moving window. This approach allows for assessing long-term changes in the intensity of HFISOs and LFISOs over the historical period from 1940 to 2014.

The analysis focuses on the JJAS period, traditionally regarded as the Indian Summer Monsoon season, but the methodology can be adapted for other seasonal or annual periods.

The analysis and results follow the next outline:

**[](climate_projections-cmip6_validation_q11:section-1)**
 * [](climate_projections-cmip6_validation_q11:section-1.1)
 * [](climate_projections-cmip6_validation_q11:section-1.2)
 * [](climate_projections-cmip6_validation_q11:section-1.3)
 * [](climate_projections-cmip6_validation_q11:section-1.4)
 * [](climate_projections-cmip6_validation_q11:section-1.5)
 * [](climate_projections-cmip6_validation_q11:section-1.6)

**[](climate_projections-cmip6_validation_q11:section-2)**
 * [](climate_projections-cmip6_validation_q11:section-2.1)
 * [](climate_projections-cmip6_validation_q11:section-2.2)
 * [](climate_projections-cmip6_validation_q11:section-2.3)


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Methodology
---
-2.1)
 * [](climate_projections-cmip6_validation_q11:section-2.2)
 * [](climate_projections-cmip6_validation_q11:section-2.3)

**[](climate_projections-cmip6_validation_q11:section-3)**
 * [](climate_projections-cmip6_validation_q11:section-3.1)
 * [](climate_projections-cmip6_validation_q11:section-3.2)
 * [](climate_projections-cmip6_validation_q11:section-3.3)
 * [](climate_projections-cmip6_validation_q11:section-3.4)
 * [](climate_projections-cmip6_validation_q11:section-3.5)
 * [](climate_projections-cmip6_validation_q11:section-3.6)
 * [](climate_projections-cmip6_validation_q11:section-3.7)


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 1. Parameters, requests and functions definition > 1.2. Define Parameters
---
In the "Define Parameters" section, various customisable options for the notebook are specified:

- The initial and ending year used for the historical period can be specified by changing the parameters `year_start` and `year_stop` (1940-2014 is chosen).
- `timeseries` sets the temporal period. For instance, selecting "JJAS" implies considering only the June, July, August, September season.
- `area` allows specifying the geographical domain of interest (NWSE). A domain including the Indian sub-continent has been selected.
- `area_timeseries` specifies the region of interest when displaying the timeseries. We have chosen the Indian Peninsula for this assessment. This domain should be equal or smaller than `area`.
- The `interpolation_method` parameter allows selecting the interpolation method when regridding is performed.
- The `chunk` selection allows the user to define if dividing into chunks when downloading the data on their local machine. Although it does not significantly affect the analysis, it is recommended to keep the default value for optimal performance.
- `variable` and `collection_id` are not customisable for this assessment and are set to 'precipitation' and 'CMIP6'. Expert users can use this notebook as a guide to create similar analyses for other variables or model sets (such as CORDEX).

Choose annual or seasonal timeseries
Interpolation method
Area to show (NWSE)
Area for the timeseries analysis (equal or smaller than area)
Chunks for download
Collection id

(climate_projections-cmip6_validation_q11:section-1.3)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 1. Parameters, requests and functions definition > 1.3. Define input models and output parameters
---
The following climate analyses are performed considering a subset of GCMs from CMIP6.

The selected CMIP6 models have available both the historical and SSP5-8.5 experiments.

Ranges of the bands used to filter and get the ISOs (Intra-Seasonal Oscillations)

(climate_projections-cmip6_validation_q11:section-1.4)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 1. Parameters, requests and functions definition > 1.4. Define ERA5 request
---
Within this notebook, ERA5 serves as the reference product. In this section, we set the required parameters for the cds-api data-request of ERA5.

(climate_projections-cmip6_validation_q11:section-1.5)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 1. Parameters, requests and functions definition > 1.5. Define model requests
---
In this section we set the required parameters for the cds-api data-request.

(climate_projections-cmip6_validation_q11:section-1.6)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 1. Parameters, requests and functions definition > 1.6. Functions to cache
---
In this section, functions that will be executed in the caching phase are defined. Caching is the process of storing copies of files in a temporary storage location, so that they can be accessed more quickly. This process also checks if the user has already downloaded a file, avoiding redundant downloads.

Functions description:

- `get_grid_out` and `add_bounds` ensure the regrid is performed properly.

- The `select_timeseries` function subsets the dataset based on the chosen `timeseries` parameter.

- The `lanczos_bandpass_filter` uses the band-pass filter Lanczos to isolate the 10–20-day (30-60 day for the LFISOs) component of the precipitation.

- The `compute_rolling_variance` function calculates the rolling variance and coefficient of variation of the filtered data using a 30-year window (which are needed when displaying the time series afterwards). It also computes the variance and coefficient of variation for the entire period.

- The `compute_intraseasonal_variance` function calculates daily anomalies, filters the low and high frequency intra seasonal oscillations (by calling `lanczos_bandpass_filter`) and selects the season using the `select_timeseries` function. It then computes the rolling variance (and coefficient of variation) of the filtered data over the historical period (1940–2014) by calling `compute_rolling_variance`. The window width is 30 years, with a step size of one year. The variance and coefficient of variation is also calculated for the whole historical period.

Design FIR filter using `firwin`
Apply the filter using filtfilt for zero-phase distortion
Store results with unique names
Merge into a Dataset
Apply resample reduction if specified
Change units
get the anomalies
get the filtered data
Select the time series for both ds and ds_filtered
Compute the variance and coefficient of variation
Create the new dataset with NaN values for the calculating the variance and CV for the whole period
Handle grid output if requested

(climate_projections-cmip6_validation_q11:section-2)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 2. Downloading and processing > 2.1. Download and transform ERA5
---
In this section, the `download.download_and_transform` function from the 'c3s_eqc_automatic_quality_control' package is used to download ERA5 reference dailly data, calculate daily anomalies, filter the low and high fequency intra seasonal oscillations, select the season (e.g., "JJAS" in this case), and compute the variance and coefficient of variation of the filtered data for the entire historical period, as well as for a 30-year rolling window. The results are then cached to prevent redundant downloads and processing.

(climate_projections-cmip6_validation_q11:section-2.2)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 2. Downloading and processing > 2.2. Download and transform models
---
In this section, the `download.download_and_transform` function from the 'c3s_eqc_automatic_quality_control' package is employed to download daily data from the CMIP6 models, calculate daily anomalies, filter the low and high fequency intra seasonal oscillations, select the season ("JJAS" in this example), compute the variance and coefficient of variation of the filtered data for the entire historical period (as well as for a 30-year rolling windows), interpolate to the ERA5 grid (only when specified; otherwise, the original model grid is maintained), and cache the results to avoid redundant downloads and processing.

Original model
Interpolated model

```text
model='access_cm2'
model='bcc_csm2_mr'
model='canesm5'
model='cmcc_esm2'
model='cnrm_cm6_1_hr'
model='cnrm_esm2_1'
model='ec_earth3_cc'
model='gfdl_esm4'
model='inm_cm4_8'
model='inm_cm5_0'
model='mpi_esm1_2_lr'
model='miroc6'
model='miroc_es2l'
model='mri_esm2_0'
model='noresm2_mm'
model='nesm3'
```

(climate_projections-cmip6_validation_q11:section-2.3)=


EQC Quality Assessment: "Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability"
Dataset: projections-cmip6 [CDS]
Aspect: validation_q11  |  Category: Climate_Projections
Section: Testing the capability of CMIP6 GCMs to represent precipitation intra-seasonal variability > Analysis and results > 2. Downloading and processing > 2.3. Change some attributes
---
Drop 'bnds' dimension if it exists for each dataset and for the interpolated dataset

(climate_projections-cmip6_validation_q11:section-3)=