"ensemble initial conditions"
Hey,
I want to produce ensemble predictions from the ERA5 on my disk. The notebook has the following statement:
"To get different ensemble initial conditions from the ifs pertubation, set number to an ensemble number, between 1-50."
Can I create "ensemble initial conditions" by either adding Gaussian noise to the ERA5 data, or from Gaussian noise? please provide the details.
If not, then let me know how to produce those initial conditions from the ERA5 data?
Hi,
You can theoretically produce ensemble initial conditions by adding Gaussian noise to the ERA5 data.
However, this will not be representative of the true IFS initial condition pertubation details of which can be found in various papers.
From the CRPS Paper:
For each forecast date, a small ensemble of states is propagated forward in time via independent model instances. These
instances can either be initialised by the same atmospheric state (e.g. the ERA5 deterministic analysis) or from different
initial conditions valid for the same date and time (e.g. generated from the ERA5 ensemble of initial conditions).
ERA5 is not just a determinisitc system, there exists an ERA5 ensemble.
Additionally, running inference of ERA5 will result in reduced skill as the model is fine tuned off of IFS.
So, my suggestion would be to run off of the IFS initial conditions.
Hey,
The following direct quote from the paper suggests that the 'the same ERA5 deterministic analysis' is used 'always' during training.
"Here, we always initialise the ensemble members from the same ERA5 deterministic analysis during training. For each model instance i and forecast step, we generate an independent Gaussian noise sample ξi ∼ N (0, In)"
So, only the deterministic ERA5 data was used during pre-training phase?
Yes, sorry should have been clearer, during pretraining only ERA5 data is used.
The independent Gaussian noise sample mentioned is part of the model, and is the reason why running the AIFS-ENS on the same initial conditions will produce a spread.
Introducing your own will not assist the model, and in fact downgrade performance.
If you
want to produce ensemble predictions from the ERA5 on my disk.
you can run the AIFS-ENS 51 times with the same initial conditions, however, it will not be representative of the operational model.
Note the part of the notebook
aifs-ens-v1 is designed to be an inhererently uncertain model, meaning that for the same initial conditions, different noise is applied within the model and a different forecast will be provided.
To get a spread representative of the operational model, as I said you will need to run off the IFS initial conditions.
The notebook says
To get different ensemble initial conditions from the ifs pertubation
This is only to refer to the initial conditions, not to the ensemble of the model.
As I have not heard from you, I trust this has resolved your issue.
I will close it now, feel free to reopen if you need more assistance.