You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Script to create aersol climatology from CPLHIST spinup simulations #4285
For CMIP6 we had scripts to take CPLHIST data from simulations and average into either a climatology for 1850 or into time-smoothed historical or future scenarios. We need the same kind of thing for aerosols for CMIP7.
The climatology script will be adapted in the future to work with historical and future scenarios, as well as doing an ensemble average over multiple CESM simulations. For historical, there were 5 year averages done, with a linear interpolation between those for each year. Adding the bit about ensemble average will be straight-forward, but perhaps a running average should be done, rather than fixed five year bins? I looked at the existing code that @olyson and @tilmes had, both of which were NCL, so need to be replaced and neither quite did what was needed. Since, this will be adapted in the future using LDF climatology isn't quite right either.
Hi Eric,
Are you talking about the stratospheric aerosol and ozone forcing files we have been creating?
We have a new Python script, and we used 5-day instantaneous output to create these forcings. I am not sure about scripts that produced 5-year averages, but maybe this has been done by Mike Mills in the past?
14. Hi Eric,
Are you talking about the stratospheric aerosol and ozone forcing files we have been creating?
We have a new Python script, and we used 5-day instantaneous output to create these forcings. I am not sure about scripts that produced 5-year averages, but maybe this has been done by Mike Mills in the past?
Yes, I had also hoped I could use the 5-day averages you are using for input to CAM. And originally tried using them, but found out I couldn't. The aerosol file being used for CAM specifies different things, and more general things about aerosols, rather than the aerosol deposition (for organic and black carbon and dust) at the surface. The stratospheric aerosol and ozone forcing file does have Ozone that we could extract out just the surface layer, but that's a different need,. And Ozone is only an option not something on by defaultt -- so it's not a priority now.
if there is a different file I should use, please let me know.
Also if you have a new Python script for doing these sorts of things, let me know where it is so I can look at it. I'd prefer to NOT have LMWG have to support creating climatologies of these aerosol deposition fields if possible. So if you now have a modern more comprehensive supported script for this that we could use, that sounds better to me. But, at least looking at it would be good to see if we could use it, or borrow ideas from it.
So let me know if I'm missing something, or looking at the wrong file(s). I looked at the latest case (at the time) from CESM_dev to see the files input to CAM and I only saw the one above. But, I totatally could have got it wrong.
to follow something similar to the previous naming convention. Let me know if it seems out of place, or if I should make changes to it's contents. For now it has what we need for aerosol deposition for DATM.
@ekluzek Yes, I am adding @shawnusaf here, not sure if he can see this. We also produce 3-D monthly tracer files that may be more useful. We could add other variables, especially for the monthly output, in our processing script; would that be helpful? If you prefer 5-day zonal mean files, we could also consider adding outher output when we run the model.
@ekluzek I am not sure if the location makes sense, since this is includes couple information , and not stratospheric aerosols.
We could discuss with others where to put this. However, again, we do have a program that can help next time, I think.
@ekluzek and @Tlimes, I have the latest version of our scripts for generating forcings files here : /glade/u/home/shawnh/Forcings/V10.2_DEV. As the directory name suggests, it is still a development version, but I think maybe it would have the capabilities that it sounds like you might need. It generates forcings from time series files and can apply smoothing using rolling averages along user-selected dimensions, etc, and it could probably be adapted to generate just about any climatology or transient file without a significant amount of effort.
22. @ekluzek I am not sure if the location makes sense, since this is includes couple information , and not stratospheric aerosols.
We could discuss with others where to put this. However, again, we do have a program that can help next time, I think.
@tilmes and @shawnusaf yes let me know where to put it. The previous files were here:
$CSMDATA/atm/cam/chem/trop_mozart_aero/aero/
which doesn't seem the right place to go, since the chemistry is different.
Since, it's CPLHIST data it could go under:
$CSMDATA/cplhist/cesm3_0/$CASENAEM
But, eventually these files will be ensemble averages of multiple cases, and we don't want it organized by casename as much as all the aerosol_dep files together.
So I could do something like:
$CSMDATA/cplhist/cesm3_0/aerosol_dep_clim
or make a subdirectory underneath ozone_strat_aero?
$CSMDATA/atm/cam/ozone_strataero//aerosol_dep
Or do something similar with where the ndep sfiles are, so either: $CSMDATA/atm/aerosol_dep_clim
or
$CSMDATA/atm/cam/chem/aersol_dep_clim
I think my preference would be for: $CSMDATA/atm/cam/chem/aersol_dep_clim
But, I'm good with whatever CAM-Chem peeps think is the best.
Thanks for pointing me to that script. I took a look at process_forcings.py which has two parts for me to pay attention to: xr_avg_rec and NDEP functions, and possibly xr_smooth* functions if at some point we were to decide to use smoothing. I'd need to adapt the NDEP function to work for Aerosold deposition. It doesn't handle CPLHIST files, and I'd need to adapt it for that. But, it is doing similar things, using NCO functions and using concurrent.futures for some of the multiprocessing. That was one of the things I wanted to check that the same kind of processing is being done and it is. I also took a look at the metadata, and it makes sense for me to at least use some of the same metadata as you have there.
For us to really use it in our workflow though, we'd want to see it in a github repo. You could start it as a github repository under your personal github page. But, if it's going to be kept it should likely transition to a repository under either NCAR or ESCOMP. I can help with that part, if that becomes something we want to do.
It looks like the main reason to adapt your script for this use is if we want to also do 5-day a climatology rather than monthly, and/or if we want to apply smoothing as you have there. And then of course to have common tools that are reused.
Activity
tilmes commentedon Oct 28, 2026
Hi Eric,
Are you talking about the stratospheric aerosol and ozone forcing files we have been creating?
We have a new Python script, and we used 5-day instantaneous output to create these forcings. I am not sure about scripts that produced 5-year averages, but maybe this has been done by Mike Mills in the past?
ekluzek commentedon Oct 28, 2026
Hi @tilmes! Thanks so much for chiming in!
Yes, I had also hoped I could use the 5-day averages you are using for input to CAM. And originally tried using them, but found out I couldn't. The aerosol file being used for CAM specifies different things, and more general things about aerosols, rather than the aerosol deposition (for organic and black carbon and dust) at the surface. The stratospheric aerosol and ozone forcing file does have Ozone that we could extract out just the surface layer, but that's a different need,. And Ozone is only an option not something on by defaultt -- so it's not a priority now.
Here's the storyline here:
ESCOMP/CDEPS#389
One example file I tried to use was:
$DIN_LOC_ROOT/atm/cam/ozone_strataero/ozone_strataero_b.e30_alpha07c_cesm.B1850C_MTt4s.ne30_t232_wgx3.251.001_1850climo_zm_5day_c251204.ncif there is a different file I should use, please let me know.
Also if you have a new Python script for doing these sorts of things, let me know where it is so I can look at it. I'd prefer to NOT have LMWG have to support creating climatologies of these aerosol deposition fields if possible. So if you now have a modern more comprehensive supported script for this that we could use, that sounds better to me. But, at least looking at it would be good to see if we could use it, or borrow ideas from it.
So let me know if I'm missing something, or looking at the wrong file(s). I looked at the latest case (at the time) from CESM_dev to see the files input to CAM and I only saw the one above. But, I totatally could have got it wrong.
Thanks so much.
ekluzek commentedon Oct 28, 2026
@tilmes I put my file here:
$CSMDATA/atm/cam/ozone_strataero/aerosol_dep_MTso.monthly_clim_1850_ne30pg3_CMIP7_CESM3_c20261028.nc
to follow something similar to the previous naming convention. Let me know if it seems out of place, or if I should make changes to it's contents. For now it has what we need for aerosol deposition for DATM.
slevis-lmwg commentedon Oct 30, 2026
Closing with b4b-dev merge to master.
tilmes commentedon Oct 30, 2026
@ekluzek Yes, I am adding @shawnusaf here, not sure if he can see this. We also produce 3-D monthly tracer files that may be more useful. We could add other variables, especially for the monthly output, in our processing script; would that be helpful? If you prefer 5-day zonal mean files, we could also consider adding outher output when we run the model.
tilmes commentedon Nov 2, 2026
@ekluzek I am not sure if the location makes sense, since this is includes couple information , and not stratospheric aerosols.
We could discuss with others where to put this. However, again, we do have a program that can help next time, I think.
shawnusaf commentedon Nov 2, 2026
@ekluzek and @Tlimes, I have the latest version of our scripts for generating forcings files here : /glade/u/home/shawnh/Forcings/V10.2_DEV. As the directory name suggests, it is still a development version, but I think maybe it would have the capabilities that it sounds like you might need. It generates forcings from time series files and can apply smoothing using rolling averages along user-selected dimensions, etc, and it could probably be adapted to generate just about any climatology or transient file without a significant amount of effort.
ekluzek commentedon Nov 2, 2026
@tilmes and @shawnusaf yes let me know where to put it. The previous files were here:
$CSMDATA/atm/cam/chem/trop_mozart_aero/aero/which doesn't seem the right place to go, since the chemistry is different.
Since, it's CPLHIST data it could go under:
$CSMDATA/cplhist/cesm3_0/$CASENAEMBut, eventually these files will be ensemble averages of multiple cases, and we don't want it organized by casename as much as all the aerosol_dep files together.
So I could do something like:
$CSMDATA/cplhist/cesm3_0/aerosol_dep_climor make a subdirectory underneath ozone_strat_aero?
$CSMDATA/atm/cam/ozone_strataero//aerosol_dep
Or do something similar with where the ndep sfiles are, so either:
$CSMDATA/atm/aerosol_dep_climor
$CSMDATA/atm/cam/chem/aersol_dep_clim
I think my preference would be for: $CSMDATA/atm/cam/chem/aersol_dep_climBut, I'm good with whatever CAM-Chem peeps think is the best.
ekluzek commentedon Nov 2, 2026
Thanks for pointing me to that script. I took a look at process_forcings.py which has two parts for me to pay attention to: xr_avg_rec and NDEP functions, and possibly xr_smooth* functions if at some point we were to decide to use smoothing. I'd need to adapt the NDEP function to work for Aerosold deposition. It doesn't handle CPLHIST files, and I'd need to adapt it for that. But, it is doing similar things, using NCO functions and using concurrent.futures for some of the multiprocessing. That was one of the things I wanted to check that the same kind of processing is being done and it is. I also took a look at the metadata, and it makes sense for me to at least use some of the same metadata as you have there.
For us to really use it in our workflow though, we'd want to see it in a github repo. You could start it as a github repository under your personal github page. But, if it's going to be kept it should likely transition to a repository under either NCAR or ESCOMP. I can help with that part, if that becomes something we want to do.
It looks like the main reason to adapt your script for this use is if we want to also do 5-day a climatology rather than monthly, and/or if we want to apply smoothing as you have there. And then of course to have common tools that are reused.
ekluzek commentedon Nov 2, 2026
It looks like useful; metadata that we should add are:
And from looking at the NDEP file, metadata that could be added: